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ABSTRACT 


In  this  thesis,  a  model  of  effectiveness  for  an  offshore  patrol  vessel  conducting  search  and 
rescue  missions  is  developed  and  described.  Beginning  with  a  brief  overview  of  work 
done  by  colleagues  from  the  University  of  Genoa,  Italy,  as  well  as  documents  currently  in 
use  by  the  United  States  Navy  and  Coast  Guard  for  search  and  rescue,  this  thesis  provides 
a  link  between  physical  ship  design  factors  and  the  operational  effectiveness  of  a  search 
and  rescue  mission. 

The  methodology  involved  developing  a  search  model,  then  using  an  enhanced 
experimental  design  to  explore  how  operational  noise  factors,  along  with  physical  ship 
characteristics,  impact  the  effectiveness  of  search  and  rescue.  Those  characteristics 
include  the  ship’s  maximum  speed,  the  number  of  helicopters  onboard,  and  the  number  of 
unmanned  aerial  vehicles  onboard.  Operational  noise  factors  include  the  visibility,  the 
direction  of  the  wind,  the  maximum  speeds  as  well  as  the  search  speeds  of  the  other 
search  entities,  the  distance  to  the  last  known  datum,  the  uncertainty  radius  of  the  last 
known  datum,  and  other  environmental  factors.  Four  metamodels  are  then  developed  to 
express  which  factors  have  the  greatest  impact  on  the  performance  of  the  ship  as  a 
function  of  cumulative  probability  detection  threshold  in  a  search  and  rescue  mission. 
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EXECUTIVE  SUMMARY 


Historically,  ship  builders  have  designed  ships  to  optimize  the  perfonnance  of 
their  primary  missions.  Search  and  rescue  (SAR),  has  traditionally  been  a  secondary 
mission  in  design. 

Beginning  with  a  brief  overview  of  work  done  by  colleagues  from  the  University 
of  Genoa,  Italy,  as  well  as  documents  currently  in  use  by  the  United  States  Navy  and 
Coast  Guard  for  search  and  rescue,  this  thesis  explores  the  link  between  physical  ship 
characteristics  and  operational  effectiveness  by  building  and  exploring  the  Naval 
Postgraduate  School  Search  (NPSS)  model.  In  this  effort,  the  impact  of  carrying  more 
helicopters,  unmanned  aerial  vehicles,  and/or  increasing  the  ship’s  maximum  speed  can 
be  clearly  seen  in  the  measure  of  effectiveness  for  the  SAR  mission. 

Additionally,  this  thesis  incorporates  operational  noise  factors  such  as  visibility, 
wind  direction,  the  maximum  speeds  as  well  as  the  search  speeds  of  the  other  search 
entities,  distance  to  the  last  known  datum,  the  uncertainty  radius  of  the  last  known  datum, 
and  other  environmental  factors,  in  an  attempt  to  determine  if  these  ship  designs  perform 
robustly  across  the  myriad  of  noise  factors. 

Four  metamodels  are  then  developed  to  express  which  factors  have  the  greatest 
impact  on  the  performance  of  the  ship  for  four  different  cumulative  probability  detection 
thresholds  in  the  SAR  mission.  The  NPSS  results  show  that  the  operational  effectiveness 
varies  substantially  for  different  ship  configurations,  but  that  there  is  still  a  great  deal  of 
variability  that  the  metamodels  cannot  capture.  The  maximum  ship  speed  is  the  dominant 
factor  for  thresholds  of  25%,  50%,  and  75%,  but  that  the  availability  of  at  least  one 
unmanned  aerial  vehicle  is  the  most  important  if  a  95%  threshold  is  required. 

The  results  from  the  NPSS  model  are  compared  to  and  contrasted  with  those  of 
the  Italian  model  for  the  restricted  set  of  circumstances  explored  in  the  earlier  Italian 
study.  Although  the  two  models  have  different  measures  of  effectiveness  and  therefore 
cannot  be  directly  compared,  the  metamodels  show  that  the  factors  have  similar 
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importance  in  the  two  models.  For  the  restricted  set  of  search  operations  examined  in  the 
Italian  study,  the  output  of  both  the  Italian  model  and  the  NPSS  model  are  very 
predictable. 

These  results  contribute  to  a  larger  project  that  aims  at  developing  a  methodology 
for  evaluating  the  operational  effectiveness  of  offshore  patrol  vessels  for  a  variety  of 
missions  before  proceeding  to  the  detailed  design  of  these  units. 
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I.  INTRODUCTION 


A.  BACKGROUND 

Throughout  history,  the  great  nations  have  been  those  which  controlled  the  seas. 
From  the  ancient  times  of  Persia  to  the  World  War  II  days  of  Japan,  loss  of  sea  power  has 
caused  many  nations  to  fail  (NAVEDTRA,  2012).  “Control  of  the  seas  means  security. 
Control  of  the  seas  can  mean  peace.  Control  of  the  seas  can  mean  victory.  The  United 
States  must  control  the  seas  if  it  is  to  protect  your  security”  (Kennedy,  1963).  The  United 
States  Navy  has  been  traditionally  identified  with  the  operation  of  iconic  capital  ships 
such  as  battleships  and  aircraft  carriers.  These  blue  water  vessels  provide  the  sustainable 
logistic  reach,  allowing  a  persistent  presence  all  around  the  world.  Although  these  ships 
have  been  predominantly  used  in  power  projection  and  deterrence  to  maintain  security, 
these  capital  ships  have  also  been  the  primary  means  for  the  U.S.  to  accomplish  all  of  its 
secondary  missions  as  well.  Even  though  these  large  ships  were  not  initially  designed 
with  some  secondary  missions  as  a  requirement,  in  an  attempt  to  compensate,  ships  are 
retrofitted  to  accommodate  these  missions.  The  U.S.  Landing  Helicopter  Dock  (LHD) 
and  Landing  Helicopter  Assault  (LHA)  types  of  ships  for  example,  were  not  configured 
to  conduct  the  Search  and  Rescue  (SAR)  mission.  Crew  berthing,  work  spaces,  even  deck 
parking  for  the  SAR  helicopters,  were  not  initially  incorporated  in  the  design  of  these 
ships,  and  currently  are  accommodated  via  means  of  retrofitting.  If  smaller,  cheaper  ships 
could  be  designed  with  these  secondary  naval  duties  as  their  primary  requirement,  clearly 
they  would  perform  better  than  the  current  blue  water  vessels  for  their  respective 
missions.  This  concept  has  been  the  objective  of  not  only  the  U.S.,  but  many  countries 
worldwide. 

For  most  countries,  naval  duties  consist  of  SAR,  Maritime  Interdiction  Operation 
(MIO),  fishery  protection,  pollution  control,  fire-fighting,  salvage  operations,  anti-surface 
warfare  (ASuW),  anti-air  warfare  (AAW),  counter-narcotics,  humanitarian  operations 
and  exclusive  economic  zone  (EEZ)  patrol.  To  perform  these  duties,  many  different 
designs  for  Offshore  Patrol  Vessels  (OP Vs)  have  been  created,  each  to  conduct  specific 
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tasks,  with  the  designs  being  based  on  the  experience  of  the  naval  architect.  At  least  30 
countries  are  known  to  have  a  total  of  89  OPVs  currently  on  order,  while  planning  for 
another  98  at  a  value  of  over  $15  billion  (Offshore  Patrol  Vessel  Sector  Report,  2010). 

The  OPV  type  chosen  depends  on  that  country’s  specific  naval  requirements. 
These  requirements  can  form  based  on  the  country’s  geographic  location,  political 
aspirations  and/or  intended  role  of  its  naval  force  in  the  world.  However,  the  majority  of 
OPV  programs  are  of  a  cheaper  basic  patrol  vessel  that  can  be  used  in  a  variety  of  roles. 

1.  Model-Based  Ship  Design 

The  Office  of  Naval  Research  (ONR)  is  working  with  Naval  Postgraduate  School 
(NPS)  to  support  a  group  of  Italian  research  colleagues  in  assessing  their  operational 
effectiveness  models  (OEMs)  representing  the  OPV’s  perfonnance  capabilities  in  a  set  of 
different  naval  operational  scenarios.  These  scenarios  of  interest  include  missions  such  as 
ASuW,  AAW,  MIO,  and  SAR. 

The  overall  project  aims  at  developing  a  methodology  for  evaluating  the 
operational  effectiveness  of  OPVs  before  proceeding  to  the  detailed  design  of  these  units. 
In  particular,  this  methodology  should  be  the  basis  for  the  development  of  the  OEM,  for 
quickly  analyzing  the  impact  of  different  choices  in  unit  requirements,  quantified  in  terms 
of  Measure  of  Effectiveness  (MOE).  The  OEMs  will  work  jointly  with  another  module, 
the  Ship  Synthesis  Model  (SSM),  in  order  to  properly  evaluate  the  features  that  units 
must  show  before  releasing  them  to  the  successive  detailed  design  phase.  As  part  of  this 
overall  project,  this  thesis  will  specifically  focus  on  the  SAR  mission,  which  is  a  well- 
known  naval  operation.  While  there  is  a  breadth  of  possible  SAR  operations  and 
scenarios,  this  thesis  focuses  on  a  ship  responding  to  a  distress  signal. 
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2. 


Search  and  Rescue 


The  United  States  Defense  Department  defines  SAR  as  an  operation  normally 
coordinated  by  a  Rescue  Coordination  Center  (RCC)  or  rescue  sub-center,  using  available 
personnel  and  facilities  to  locate  persons  in  distress  and  deliver  them  to  a  place  of  safety 
(England,  2006). 

There  are  various  forms  of  SAR:  ground  SAR  is  traditionally  associated  with 
wilderness  zones,  and  urban/suburban  environments  (Urban  Search  and  Rescue  [US&R] 
2009);  combat  SAR  occurs  when  SAR  operations  are  carried  out  during  war  or  near 
combat  zones  (About.com  U.S.  Military,  2009).  Lastly,  air-sea  rescue  (ASR),  which 
refers  to  the  combined  use  of  aircraft  (helicopters,  and  fixed-wing  unmanned  aerial 
vehicles  (UAVs))  and  surface  vessels  to  search  for  and  recover  survivors  of  aircraft 
downed  at  sea  as  well  as  passengers  of  vessels  in  distress,  are  the  fonn  of  SAR  that  is 
mostly  discussed  in  this  paper. 

The  main  purpose  of  SAR  operations  for  the  U.S.  Navy  is  to  save  lives,  the  most 
expensive  component  of  the  Navy.  U.S.  maritime  SAR  is  conducted  predominantly  by 
the  U.S.  Coast  Guard  (USCG),  and  the  U.S.  Navy. 

The  USCG  maintains  a  wide  variety  of  SAR  resources,  specifically  dedicated  to 
conduct  maritime  SAR  throughout  the  U.S.  Medium-range  SAR  involves  fixed-wing 
aircraft  such  as  the  HU-25,  and  rotary- wing  aircraft  such  as  the  HH-60  and  HH-65.  In 
addition  to  aircraft,  the  USCG  utilizes  378  foot  high  endurance  cutters  (WHECs),  1 80- 
270  foot  medium  endurance  cutters  (WMECs)  and  80-110  foot  patrol  boats  (WPB) 
(United  States  National  Search  and  Rescue  Supplement  2000). 

The  United  States  Navy  (USN)  maintains  extensive  numbers  and  types  of  aircraft, 
including  both  fixed  and  rotary  wing.  The  P-3  is  used  for  long  range  missions,  the  S-3 
and  E-2  are  carrier  based  fixed-wing  aircraft,  and  the  H-60  rotary  wing  type  is  utilized 
onboard  various  different  vessels.  The  vessels  most  often  utilized  by  the  USN  include 
destroyers  (DD)  and  fast  frigates  (FF)  for  surface  search,  nuclear-powered  aircraft 
carriers  (CVNs),  landing  helicopter  docks  (LHDs)  and  landing  helicopter  assaults 
(LHAs)  for  air  search,  and  submarines  for  subsurface  search. 
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Currently,  the  USN  conducts  SAR  operations  in  accordance  with  the  Navy  Search 
and  Rescue  Tactical  Infonnation  Document  (Office  of  the  Chief  of  Naval  Operations, 
1997)  which  consists  of  lookup  tables,  charts,  graphs,  and  step-by-step  instructions  on 
how  to  conduct  SAR.  The  infonnation  used  to  build  this  document  stems  from  the  United 
States  National  Search  and  Rescue  Supplement  (National  Search  and  Rescue  Committee, 
2000),  which  has  been  constructed  through  actual  search  scenario  experiments. 

As  far  as  modeling  SAR,  there  are  different  approaches  that  can  be  taken. 
Simulation  is  an  approach  that  can  be  used  to  mimic  movement,  patterns,  and  visual 
detection  ranges  of  a  typical  search  scenario.  Another  approach  is  to  make  use  of  the 
tables  that  populate  the  SAR  supplement,  to  calculate  estimates  for  a  given  SAR  scenario 
directly. 

B.  THESIS  OBJECTIVES  AND  ORGANIZATION 

In  this  thesis,  a  model  of  SAR  operations  is  developed  using  a  Microsoft  Excel- 
based  approach  to  calculate  estimates  for  the  SAR  scenario  directly.  Once  the  model  is 
developed,  a  state-of-the-art  experimental  design  is  used  to  explore  variants  of  this  model 
in  an  efficient,  systematic  process.  The  primary  goal  is  to  answer  the  primary  and 
secondary  questions  for  ONR: 

Can  a  model  of  SAR  Operations  be  developed  using  Microsoft  Excel  to  show 
potential  mission  effectiveness  of  a  ship  design  concept  with  results  comparable  to  the 
Italian  research  team’s  simulation  results? 

Can  this  new  model  be  improved  through  more  realistic  operational 
representantion  and  explored  using  enhanced  experimental  design  techniques  in  order  to 
provide  broader  insights  than  the  results  from  the  original  Italian  model? 

While  there  is  a  breadth  of  possible  SAR  operations  and  scenarios,  this  thesis 
focuses  on  a  ship  responding  to  a  distress  signal. 
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II.  MODELING  THE  OPV  IN  SAR  OPERATIONS 


A.  INTRODUCTION 

To  develop  a  basic  SAR  model,  there  must  be  at  least  three  entities:  an  OPV,  a 
helicopter,  and  a  search  target.  In  order  to  ensure  the  model  helps  analyze  the  same 
scenario  analyzed  by  the  Italian  group,  at  the  bare  minimum,  the  same  key  assumptions 
need  to  be  considered  as  a  starting  point.  From  there,  the  idea  is  to  explore  different 
avenues  to  add  realism  while  at  the  same  time  ensuring  the  objective  of  the  model  is 
continuously  being  met,  namely  measuring  the  effectiveness  of  different  ship 
configurations  in  the  SAR  mission. 

B.  ITALIAN  MODEL 

1.  Overview  and  Assumptions 

Our  Italian  research  colleagues  from  the  University  of  Genoa,  Italy  as  mentioned 
in  Chapter  I  created  a  SAR  scenario  (shown  in  Figure  1)  in  which  an  OPV  (Offshore 
Patrol  Vessel  Sector  Report,  2010)  is  searching  for  a  fishing  boat  labeled  “FB”. 
Throughout  this  thesis,  this  model  is  referred  to  as  “the  Italian  model”.  In  this  scenario, 
the  OPV  receives  a  distress  signal,  in  which  the  fishing  boat  relays  that  it  will  imminently 
be  losing  propulsion  as  well  as  communication  capabilities.  The  OPV  also  has  the 
infonnation  on  a  last  known  datum  of  the  fishing  boat  to  be  a  distance  d  away.  This 
location  of  the  fishing  boat  at  the  time  of  the  distress  signal  is  not  known  exactly,  but  it  is 
assumed  to  be  uniformly  distributed  within  a  circle  of  radius  r  (called  the  uncertainty 
radius)  around  the  datum. 
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OPV 


- ► 

(0, 0) 


Figure  1.  Italian  model  OPV  and  datum  initial  positions  (From  Anghinolfi  et  al.,  2011) 


Figure  2.  Italian  model  fishing  boat  directional  bounds  (From  Anghinolfi  et  ah,  2011) 
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Figure  2  shows  that  additional  uncertainty  arises  because  the  target  can  move.  The 
model  assumes  the  target/FB  movement  velocity  (represented  as  V°)  remains  fixed,  and 
this  movement  will  be  limited  to  being  a  direct  function  of  the  wind  velocity: 


VD  =  EstimatedWindSpeed  ■  (^-) 

2-0  (Equation  1) 

According  to  the  Italian  group,  this  relationship  between  the  wind  speed  and 
target  speed  is  an  estimate  that  comes  from  the  Italian  SAR  manual.  The  model  also 
assumes  target  movement  directional  bounds  5  with  fixed  values  equal  to  ±31/8  radians  in 
the  direction  of  the  wind.  Additionally,  the  direction  of  the  wind  with  respect  to  the  x- 
axis  is  represented  by  the  variable  4)  (shown  in  Figure  2).  Using  the  known  maximum 
wind  velocity,  the  outer-most  coordinates  of  the  three  intercept  data  are  computed  (shown 
in  equations  2  through  5)  using  Cartesian  motion  equations  (equations  6  through  9).  Then 
by  including  the  original  datum  location,  a  search  box  is  constructed  to  encapsulate  all 
four  data  plus  each  of  their  respective  uncertainty  buffers  (shown  in  Figure  3). 


Figure  3.  Italian  model  determination  of  the  exploration  area  (From  Anghinolfi  et  ah,  201 1) 
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FB  .  .  t rFB 

x  =  xo  +t-Vx 

(Equation  2) 

S3  - 

SC 

+ 

^  o 
>> 

II 

(Equation  3) 

x  =  x°PV+t-VZV-  cos£ 

(Equation  4) 

y-yr+t-vzvsin9 

(Equation  5) 

Xj  =  x2  =  minjxo s ,  xA ,  xB ,  xc}  -  r 

(Equation  6) 

x3  =x4  =  max  {x™ ,  xA ,  xB ,  xc  J  +  r 

(Equation  7) 

T,  =  >4  =  min  { v oB  ,yA,yB,  yc }  -  r 

(Equation  8) 

y2  =y3  =  max  { y ™ » yA .  yB .  yc }  + r 

(Equation  9) 

Once  the  search  box  is  defined,  the  OPV  moves  at  maximum  ship  velocity  to  the 
closest  comer  of  the  search  box,  and  then  begins  searching  for  the  target. 

The  ship  is  designed  to  follow  a  coordinated  creeping  line  search  pattern  with  a 
single  helicopter  overhead,  once  the  ship  has  reached  the  interior  of  the  search  box.  A 
portion  of  the  search  path  is  shown  in  Figure  4.  Similarly,  the  ship  itself  will  follow  a 
larger  creeping  line  search  pattern  if  the  search  box  is  large. 


Figure  4.  Coordinated  creeping  line — single  unit  (From  Office  of  the  Chief  of  Naval 

Operations,  1997) 

In  this  model,  even  if  the  ship  or  helicopter  comes  close  to  the  target,  detection  is 
not  guaranteed.  The  detection  depends  on  the  cumulative  probability  of  detection  (CDP) 
of  the  ship  and  helicopter  using  a  detection  rate  model.  Daniel  Wagner  in  his  book 
“Naval  Operations  Analysis”  describes  how  a  detection  rate  model  can  be  used  to  obtain 
a  CDP: 

For  the  time  t  >  0,  the  CDP  at  time  t,  is  defined  to  be  the  probability  that  detection 
occurs  at  least  once  during  [0,f]: 


8 


CDP(t)  =  Pr{at  least  one  detection  occurs  no  later  than  time  t) 

CDP(t)  =  Pr{time-to-initial-detection<=t} 

Search  models  can  consist  of  a  sequence  of  discrete  glimpses,  or  they  may  be 
continuous  looking  model.  In  the  Italian  model’s  case,  the  simulation  uses  a  continuous 
looking  model  to  obtain  a  cumulative  probability  of  detection. 

With  the  assumption  that  the  detection  rate  of  the  search  entity  occurs  at  a 
detection  rate,  Y(t)  at  time  t,  the  notion  of  independence  can  be  extended  to  a  continuous 
looking  model.  Detections,  (not  necessarily  the  initial  detections),  can  now  occur  as  a 
Poisson  process,  with  a  variable  rate  parameter,  Y(t).  This  would  imply: 

For  any  time  t,  for  a  small  h  >0,  where  h  is  a  timestep, 

Pr {AtLeastOneDetectionOccursIn[t , t  +  h]}  ~  h  ■  Y(t),  and 

Vr{MoreThanOneDetectionOccursIn[t,  t  +  h]}  is  negligible  compared  to  h  ■  Y(t) 

Occurrences  of  detections  in  nonoverlapping  time  intervals  are  independent. 

With  this  modification,  the  CDP  can  be  expressed  in  terms  of  Y(t),  during  the  time 
interval  [0,t]: 

t 

— J  y(u)du 

p(t)  =  l-e°  (Equation  10) 

(Wagner,  Sanders  &  Mylander,  1999) 

The  SAR  operation  ends  once  the  ship  has  either  located  the  target,  or  completed 
a  single  search  of  the  box  without  finding  the  target.  By  simulating  this  operation  over 
10,000  runs  with  target  initial  positions  uniformly  distributed  over  the  datum  uncertainty 
radius,  the  Italian  model  can  estimate  the  probability  of  success  for  a  specific  set  of  initial 
conditions. 

2.  Italian  Model  Experiments 

The  Italian  model  is  run  using  ten  fixed  parameters  and  varying  three  discrete 
factors  shown  in  Table  1. 
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Fixed  parameter 

Value 

Datum  Uncertainty  r 

2  nm 

Wind  maximum  speed 

20  kn 

OPV  patrolling  speed 

15  kn 

OPV  acceleration 

0.1  m/s2 

W  value  of  OPV 

2  nm 

W  value  of  Helicopter 

5.8  nm 

Helicopter  speed 

70  kn 

Helicopter  maximum  flight  time 

150  min 

Helicopter  setup  time 

30  min 

Helicopter-OPV  maximum  distance 

20  nm 

Table  3  -  Fixed  parameters  of  the  SAR  scenario 


DOE  factor 

Range 

Initial  distance  d 

[50  nm,  200  nm] 

OPV  maximum  speed 

[22  kn,  40  kn] 

Helicopter  presence 

[0, 1]  (binary  value) 

Table  1.  Italian  model  SAR  scenario  factors  (From  Anghinolfi  et  al.,  2011) 

3.  Italian  Study  Findings 

After  running  the  model  with  experiments,  the  Italian  study  concludes  that  of  the 
three  variable  factors,  the  presence  of  the  helicopter  has  the  largest  impact  on  the  output 
of  the  model’s  measure  of  effectiveness,  followed  by  the  OPV’s  maximum  speed,  and 
lastly  by  the  initial  distance.  More  results  on  the  Italian  study  can  be  found  in  Chapter 
III.C. 


4.  limitations  of  the  Italian  Model  and  Study 

The  Italian  model  uses  the  CDP  of  a  ship  to  represent  SAR  perfonnance;  however 
it  does  not  seem  to  put  any  emphasis  on  limiting  time.  The  maximum  amount  of  time 
required  to  finish  a  scenario  varies  with  different  ship  configurations.  Some  scenarios  end 
relatively  quickly,  while  others  take  a  much  larger  amount  of  time  to  complete. 

If  a  ship  is  able  to  conduct  SAR  with  a  CDP  of  0.95  would  seem  close  to  optimal, 
however  if  it  requires  the  ship  two  months  of  searching  to  attain  that  cumulative 
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probability,  then  in  reality  the  fishing  boat  crew  may  not  have  survived  the  search  effort. 
This  may  mean  that  time  should  also  be  included  as  a  measure  of  effectiveness  for  SAR. 

Additionally,  the  Italian  model  solely  accounts  for  the  effect  of  a  single  SAR 
helicopter.  Expanding  the  ship  design  to  include  the  possibly  of  accommodating  multiple 
helicopters  may  allow  the  ship  to  achieve  a  higher  probability  of  detection  for  area  search 
(Pd).  Along  the  same  lines,  having  UAVs  may  prove  to  be  a  more  cost  effective  way  to 
achieve  the  same  capabilities  as  having  additional  helicopters  attached  to  the  ship. 

Lastly,  the  Italian  model  seems  overly  constrained.  Many  model  parameters  that 
would  vary  in  actual  SAR  missions  are  fixed  for  the  study;  these  parameters  will  be 
labeled  the  noise  factors.  For  example,  the  search  itself  was  hardcoded  to  find  one 
specific  target,  namely  a  single  survivor  in  the  water,  when  SAR  in  general  encompasses 
wide  ranges  for  number  of  personnel,  as  well  as  the  length  of  the  ship  to  be  rescued. 
Because  some  of  the  noise  factors  in  the  Italian  model  are  fixed,  it  is  difficult  to 
detennine  how  the  model  behaves  if  those  fixed  parameters  have  a  different  set  of  values. 

The  questions  then  arise,  “How  does  having  more  search  assets  affect  the  SAR 
mission  given  a  search  box?  Is  it  worth  having  more  than  two  search  assets  (a  helicopter 
and  a  ship)?” 

C.  ALTERNATIVE  SEARCH  MODELS 

Before  the  NPSS  model  is  presented,  a  brief  discussion  of  other  search  models 
and  methods  are  provided  below. 

1.  Cooperative  Search  Model 

An  analysis  conducted  by  Vincent  and  Rubin  (2004)  sought  to  analyze  the  effects 
of  cooperative  search  using  UAV.  The  analysis  assumed  that  each  search  unit  performed 
the  search  in  a  “cooperative  manner,”  meaning  that  each  search  unit  was  able  to 
communicate  and  coordinate  with  every  other  search  unit  in  the  execution  of  their  search 
effort.  The  cookie  cutter  detection  rule  was  used  (otherwise  known  as  the  definite  range 
law  of  detection);  specifically,  it  was  assumed  the  search  units  always  detect  targets 

within  a  specified  radial  distance.  Under  these  assumptions,  it  was  found  that  the  time 
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required  to  complete  the  search  decreased  exponentially,  as  more  and  more  search  units 
were  added  to  the  task  (Figure  5).  After  a  certain  number  of  search  units,  diminishing 
returns  are  noted  in  the  time  required  to  complete  the  search.  This  paper  provides  the 
insight  that  having  more  search  assets  allows  you  to  complete  the  searching  of  your 
search  box  in  a  shorter  period  of  time,  but  only  up  to  a  point.  Although  it  would  not  be  as 
efficient  if  the  search  was  conducted  by  independent  search  units  rather  than 
“cooperative”  search  units,  it  can  be  deduced  that  when  adding  additional  search  units 
with  respective  probabilities  of  detection  greater  than  zero,  the  cumulative  probability  of 
detection  can  only  increase  in  a  similar  fashion. 


Figure  5.  Time  to  complete  parallel-path  search  (From  Vincent  &  Rubin,  2004) 

Although  the  cooperative  search  model  utilized  a  cookie  cutter  detection  rule, 
sensors,  detectors,  and  even  observer’s  conducting  visual  search  can  come  in  a  wide 
variety  of  detection  capabilities.  Some  are  near-sighted,  some  are  far-sighted,  and  some 
see  better  with  the  left  eye  than  the  right.  An  observer’s  detection  of  a  target  depends  on 
many  variables. 
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2. 


National  SAR  Supplement 


To  avoid  specific  assumptions  concerning  the  visual  detection  function,  such  as 
the  cookie  cutter  (definite  range  law)  assumption  used  in  the  Cooperative  Search  Model, 
actual  data  can  be  used.  The  National  SAR  Supplement  contains  empirical  data  on 
effective  sweep  width  for  various  search  platforms  and  various  search  objects  under 
differing  conditions,  obtained  from  actual  search  experiments.  A  portion  of  one  such 
table  is  shown  in  Table  2  as  Uncorrected  Visual  Sweep  Width.  Given  a  target  for  search, 
SAR  aircraft  altitude,  and  the  visibility  present,  these  tables  provide  an  estimate  of  an 
uncorrected  visual  sweep  width  for  the  given  SAR  asset. 


Search  Object 

Altitude  300  Feet 

Visibility  (NM) 

Altitude  500  Feet 

Visibility  (NM) 

1 

3 

5 

10 

15 

20 

30 

1 

3 

5 

10 

15 

20 

30 

Person  in  Water* 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

0.0 

0.1 

0.1 

0.1 

0.1 

0.1 

0.1 

Raft  1  person 

0.3 

0.7 

0.9 

1.2 

1.3 

1.3 

1.3 

0.3 

0.7 

0.9 

1.2 

1.4 

1.4 

1.4 

Raft  4  person 

0.4 

0.9 

1.3 

1.7 

2.0 

2.2 

2.2 

0.4 

1.0 

1.3 

1.8 

2.0 

2.2 

2.2 

Raft  6  person 

0.1 

1.1 

1.5 

2.1 

2.5 

2.7 

2.7 

0.4 

1.1 

1.5 

2.2 

2.5 

2.8 

2.8 

Raft  8  person 

0.4 

1.2 

1.6 

2.3 

2.6 

2.9 

2.9 

0.4 

1.2 

1.6 

2.3 

2.7 

2.9 

2.9 

Raft  10  person 

0.4 

1.2 

1.7 

2.4 

2.9 

3.2 

3.2 

0.4 

1.2 

1.7 

2.5 

2.9 

3.2 

3.2 

Raft  15  person 

0.5 

1.3 

1.9 

2.7 

3.2 

3.5 

4.0 

0.5 

1.3 

1.9 

2.7 

3.3 

3.6 

4.0 

Raft  20  person 

0.5 

1.4 

2.1 

3.1 

3.7 

4.2 

4.8 

0.5 

1.5 

2.1 

3.2 

3.8 

4.2 

4.8 

Raft  25  person 

0.5 

1.5 

2.2 

3.4 

4.1 

4.6 

5.2 

0.5 

1.6 

2.3 

3.4 

4.1 

4.6 

5.3 

Power  Boat  <  1 5  ft 

0.4 

0.8 

1.1 

1.4 

1.6 

1.7 

1.7 

0.4 

0.9 

1.2 

1.5 

1.7 

1.8 

1.8 

Power  Boat  16-25  ft 

0.5 

1.6 

2.4 

3.5 

4.3 

4.8 

4.8 

0.5 

1.7 

2.4 

3.6 

4.3 

4.8 

4.8 

Power  Boat  26-40  ft 

0.6 

2.1 

3.3 

5.3 

6.6 

7.6 

9.1 

0.6 

2.1 

3.3 

5.3 

6.7 

7.7 

9.1 

Power  Boat  41-65  ft 

0.6 

2.6 

4.5 

8.1 

10.9 

13.1 

16.4 

0.6 

2.7 

4.5 

8.1 

10.9 

13.1 

16.5 

Power  Boat  66-90  ft 

0.6 

2.8 

5.0 

9.7 

13.5 

16.6 

21.6 

0.6 

2.8 

5.0 

9.8 

13.5 

16.7 

21.7 

Sail  Boat  15  ft 

0.5 

1.5 

2.2 

3.2 

3.8 

4.3 

4.3 

0.5 

1.6 

2.2 

3.2 

3.9 

4.3 

4.3 

Sail  Boat  20  ft 

0.6 

1.8 

2.6 

4.0 

4.9 

5.6 

5.6 

0.6 

1.8 

2.7 

4.0 

5.0 

5.6 

5.6 

Sail  Boat  25  ft 

0.6 

2.0 

3.1 

4.8 

6.0 

6.9 

6.9 

0.6 

2.0 

3.1 

4.9 

6.1 

7.0 

7.0 

Sail  Boat  30  ft 

0.6 

2.3 

3.6 

5.9 

7.5 

8.8 

10.6 

0.6 

2.3 

3.6 

5.9 

7.6 

8.8 

10.6 

Sail  Boat  40  ft 

0.6 

2.6 

4.3 

7.5 

10.0 

11.9 

14.8 

0.6 

2.6 

4.3 

7.6 

10.0 

11.9 

14.8 

Sail  Boat  50  ft 

0.6 

2.7 

4.6 

8.4 

11.3 

13.6 

17.3 

0.6 

2.7 

4.6 

8.4 

11.3 

13.7 

17.3 

Sail  Boat  65-75  ft 

0.6 

2.8 

4.9 

9.3 

12.7 

15.5 

20.0 

0.6 

2.8 

4.9 

9.3 

12.7 

15.5 

20.0 

Sail  Boat  76-90  ft 

0.6 

2.8 

5.1 

9.9 

13.7 

16.9 

22.1 

0.6 

2.8 

5.1 

9.9 

13.7 

17.0 

22.1 

Ship  90-1 50  ft 

0.6 

2.9 

5.4 

11.1 

15.9 

20.0 

26.9 

0.6 

2.9 

5.4 

11.1 

15.9 

20.1 

26.9 

Ship  150-300  ft 

0.6 

3.0 

5.7 

12.5 

18.8 

24.7 

34.8 

0.6 

3.0 

5.7 

12.5 

18.9 

24.7 

34.8 

Ship  >  300  ft 

0.7 

3.0 

5.8 

13.2 

20.6 

27.9 

41.4 

0.7 

3.0 

5.8 

13.2 

20.6 

27.9 

41.4 

Table  2.  Uncorrected  visual  sweep-width  for  fixed-wing  aircraft  (From  National 

Search  and  Rescue  Committee,  2000) 


Similarly,  tables  for  ships  conducting  SAR  are  also  available  in  the  supplement; 
given  a  target  to  search,  the  length  of  the  ship  conducting  SAR,  and  the  visibility,  the 
sweep  width  can  be  estimated  (Table  3). 


13 


Search  Object 

Vessel  SRU  (90'  WPB) 

Visibility  (NM) 

Small  Boat  SRU  (41'  UTB) 

Visibility  (NM) 

1 

3 

5 

10 

15 

20 

1 

3 

5 

10 

15 

20 

Person  in  Water* 

0.3 

0.4 

0.5 

0.5 

0.5 

0.5 

0.2 

0.2 

0.3 

0.3 

0.3 

0.3 

Raft  1  person 

0.9 

1.8 

2.3 

3.1 

3.4 

3.7 

0.7 

1.3 

1.7 

2.3 

2.6 

2.7 

Raft  4  person 

1.0 

2.2 

3.0 

4.0 

4.6 

5.0 

0.7 

1.7 

2.2 

3.1 

3.5 

3.9 

Raft  6  person 

1.1 

2.5 

3.4 

4.7 

5.5 

6.0 

0.8 

1.9 

2.6 

3.6 

4.3 

4.7 

Raft  8  person 

1.1 

2.5 

3.5 

4.8 

5.7 

6.2 

0.8 

2.0 

2.7 

3.8 

4.4 

4.9 

Raft  10  person 

1.1 

2.6 

3.6 

5.1 

6.1 

6.7 

0.8 

2.0 

2.8 

4.0 

4.8 

5.3 

Raft  1 5  person 

1.1 

2.8 

3.8 

5.5 

6.5 

7.2 

0.9 

2.2 

3.0 

4.3 

5.1 

5.7 

Raft  20  person 

1.2 

3.0 

4.1 

6.1 

7.3 

8.1 

0.9 

2.3 

3.3 

4.9 

5.8 

6.5 

Raft  25  person 

1.2 

3.1 

4.3 

6.4 

7.8 

8.7 

0.9 

2.4 

3.5 

5.2 

6.3 

7.0 

Power  Boat  <  1 5  ft 

0.5 

1.1 

1.4 

1.9 

2.1 

2.3 

0.4 

0.8 

1.1 

1.5 

1.6 

1.8 

Power  Boat  16-25  ft 

1.0 

2.0 

2.9 

4.3 

5.2 

5.8 

0.8 

1.5 

2.2 

3.3 

4.0 

4.5 

Power  Boat  26-40  ft 

1.1 

2.5 

3.8 

6.1 

77 

8.8 

0.8 

1.9 

2.9 

4.7 

5.9 

6.8 

Power  Boat  41-65  ft 

1.2 

3.1 

5.1 

9.1 

12.1 

14.4 

0.9 

2.4 

3.9 

7.0 

9.3 

11.1 

Power  Boat  66-90  ft 

1.2 

3.2 

5.6 

10.7 

14.7 

18.1 

0.9 

2.5 

4.3 

8.3 

11.4 

14.0 

Sail  Boat  15  ft 

1.0 

1.9 

2.7 

3.9 

4.7 

5.2 

0.8 

1.5 

2.1 

3.0 

3.6 

4.0 

Sail  Boat  20  ft 

1.0 

2.2 

3.2 

4.8 

5.9 

6.6 

0.8 

1.7 

2.5 

3.7 

4.6 

5.1 

Sail  Boat  25  ft 

1.1 

2.4 

3.6 

5.7 

7.0 

8.1 

0.9 

1.9 

2.8 

4.4 

5.4 

6.3 

Sail  Boat  30  ft 

1.1 

2.7 

4.1 

6.8 

8.6 

10.0 

0.9 

2.1 

3.2 

5.3 

6.6 

Sail  Boat  40  ft 

1.2 

3.0 

4.9 

8.5 

11.2 

13.3 

0.9 

2.3 

3.8 

6.6 

8.6 

10.3 

Sail  Boat  50  ft 

1.2 

3.1 

5.2 

9.4 

12.5 

15.0 

0.9 

2.4 

4.0 

7.3 

9.7 

11.6 

Sail  Boat  65-75  ft 

1.2 

3.2 

5.5 

10.2 

13.9 

16.9 

0.9 

2.5 

4.2 

7.9 

10.7 

13.1 

Sail  Boat  76-90  ft 

1.2 

3.3 

5.7 

10.8 

15.0 

18.4 

0.9 

2.5 

4.4 

8.3 

11.6 

14.2 

Ship  90-150  ft 

1.8 

3.3 

6.0 

12.0 

17.1 

21.5 

1.4 

2.5 

4.6 

9.3 

13.2 

16.6 

Ship  150-300  ft 

1.8 

3.4 

6.3 

13.4 

20.1 

26.0 

1.4 

2.6 

4.9 

10.3 

15.5 

20.2 

Ship  >  300  ft 

1.8 

3.4 

6.4 

14.1 

21.8 

29.2 

1.4 

2.6 

4.9 

10.9 

16.8 

22.5 

Table  3.  Uncorrected  visual  sweep-width  for  vessels  and  boats  (From  National 

Search  and  Rescue  Committee,  2000) 


From  Table  2  and  3,  it  is  apparent  that  visibility  (Vis)  and  the  target  type  (Target) 
are  factors  that  affect  the  overall  sweep  width  of  the  vessel/fixed-wing  aircraft.  If 
visibility  is  poor,  it  is  more  difficult  to  locate  a  target  than  if  visibility  is  good.  Tables  2 
and  3  show  that  as  the  visibility  decreases,  so  does  the  uncorrected  sweep  width. 

D.  NPSS  MODEL  DEVELOPMENT 

1.  Overall  Approach 

The  interest  of  this  thesis  is  to  build  an  Excel  model  called  the  Naval  Postgraduate 
School  Search  (NPSS)  model  to  address  some  of  the  limitations  identified  in  section  B,  to 
incorporate  the  option  of  having  multiple  aerial  search  entities  of  rotary  and  fixed-wing 
configurations,  to  ensure  noise  factors  and  other  physical  ship  factors  are  included  and 
varied  efficiently  in  the  model  using  a  state-of-the-art  experimental  design,  again  with  the 
goal  to  answer  the  primary  and  secondary  questions.  The  benefit  of  incorporating 
additional  noise  factors  could  potentially  lead  to  a  more  robust  model  and  in  turn,  ship 
design. 

The  NPSS  model  delineates  between  two  types  of  modeling  factors;  decision 
factors  and  noise  factors.  Decision  factors  refer  to  physical  ship  design  characteristics 
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that  a  ship  builder  can  “decide”  on,  to  include  or  exclude  in  the  construction  of  a  ship. 
Noise  factors  are  factors  in  a  realistic  search  scenario  over  which  there  is  no  control; 
environmental  conditions,  visibility,  and  target  are  examples  of  noise  factors.  A  complete 
list  of  the  decision  factors  and  noise  factors  can  be  found  in  Table  5  in  Chapter  II. 

2.  OPV 

Like  the  Italian  experiment,  the  NPSS  has  a  single  OPV.  The  maximum  speed  of 
the  ship  is  a  factor  labeled  ShipMax.  The  experiment  explores  a  large  number  of  values, 
as  well  as  a  larger  range  of  speeds,  which  includes  the  Italian  experiment’s  range  as  a 
subset  of  itself.  Additionally,  the  NPSS  experiment  incorporates  a  factor  the  ship’s  search 
speed  (SSS).  This  is  the  speed  at  which  the  ship  will  search  once  it  is  within  the  search 
box.  The  search  speed  can  be  less  than  the  maximum  speed. 

3.  Helicopter 

The  Italian  model  has  incorporated  a  hardcoded  distance  requirement  that 
restrains  a  helicopter  from  flying  farther  than  a  given  maximum  distance  from  the  ship. 
The  reason  for  this  constraint  is  to  model  the  necessity  for  communication  and  directional 
guidance  with  the  OPV. 

With  cheap  modern  global  positioning  guidance  systems  and  satellite 
radios/receivers,  and  assuming  the  helicopter  has  these  systems  installed,  the  need  for  this 
distance  constraint  can  be  eliminated.  Although  this  is  a  deviation  from  the  Italian  model, 
such  an  assumption  has  tremendous  simplifying  effects  for  modeling  purposes. 

In  the  NPSS  model,  the  helicopter  may  fly  its  flight  path  once  the  ship  has  entered 
the  search  box,  regardless  of  how  far  the  ship  may  be  from  the  helicopter.  With  this 
assumption,  and  given  the  only  ship-to-helicopter  interaction  occurs  when  the  target  is 
found,  the  search  pattern  of  the  helicopter  and  the  search  pattern  of  the  ship  can  be 
assumed  to  be  independent  of  one  another.  Search  patterns  are  discussed  in  further  detail 
in  Section  C.l  1. 
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4. 


Use  of  Multiple  Detectors 


Using  the  ideas  from  the  Cooperative  Search  model  discussed  in  Chapter  I,  it 
would  seem  that  adding  multiple  detectors  to  the  NPSS  model  may  reveal  the  possibility 
of  tradeoffs  in  the  MOE.  It  is  trivial  that  with  the  addition  of  search  detectors,  each  with  a 
probability  of  detection  greater  than  zero,  the  aggregate  detection  rate  will  increase.  Since 
there  is  nothing  constraining  a  ship  configuration  from  having  the  maximum  number  of 
detectors,  it  is  also  trivial  that  the  best  perfonnance  will  come  from  the  ship  configuration 
with  the  maximum  number  of  detectors.  What  will  be  interesting  to  analyze  however,  is 
“how  much  of  an  effect”  this  aggregate  detection  rate  increase  has  on  the  overall  MOE. 

5.  Use  of  Unmanned  Aerial  Vehicles 

Stemming  from  the  idea  that  having  multiple  detectors  would  offer  an  interesting 
trade-study,  UAVs  can  be  looked  at  as  the  cheaper  alternative  or  complement  to  the 
helicopter.  UAVs  in  general,  tend  to  be  smaller  than  helicopters,  which  in  turn  can  mean 
less  manpower  necessary  to  operate  and  maintain  the  aircraft,  smaller  requirement  for 
storage  of  aircraft  and  equipment  onboard  the  ship,  and  a  lower  fuel  expenditure  for  the 
aircraft.  These  considerations  could  directly  influence  the  ship  design.  However,  the 
greatest  advantage  is  that  the  aircraft  is  unmanned  so  the  human  risk  is  removed  from  the 
search  aspect  of  the  mission. 

6.  Target,  Visibility,  Environmental  Conditions,  and  Air  Speed 

In  the  Italian  model,  the  search  target  is  assumed  to  be  a  fishing  boat  that  sends 
out  a  final  distress  signal  to  the  search  entities,  and  loses  all  future  communication 
capabilities.  Although  only  one  type  of  target  is  used  in  the  Italian  model,  results 
obtained  from  the  NPSS  model  may  prove  to  be  more  robust  if  the  NPSS  model  is  able  to 
incorporate  and  account  for  an  array  of  possible  targets,  which  may  provide  better 
insights  as  to  what  set  of  targets  a  given  ship  would  be  ideal  to  conduct  SAR. 
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In  the  Italian  study,  these  environmental  factors  were  all  fixed  at  specific  values. 
By  varying  these  factors  in  the  NPSS  experiment,  it  will  present  insights  on  how  sensitive 
the  relationships  amongst  the  factors  in  the  MOE  are  to  the  presence  of  variable 
environmental  conditions. 

Conveniently,  the  effects  factors  such  as  visibility,  altitude,  and  target  type  have 
on  a  searcher’s  visual  sweep  width  have  been  tabularized  in  the  National  SAR 
Supplement  as  the  “Uncorrected  Sweep  Width”.  These  uncorrected  sweep  width  values 
are  then  “corrected”  by  a  correction  factor  that  allows  the  sweep  width  to  account  for 
environmental  weather  conditions,  as  well  as  the  air  speed  the  searcher  is  traveling  at,  and 
is  known  as  the  Corrected  Sweep  Width  W. 

The  use  of  sweep  width  in  the  search  and  detection  problem  was  established  by  B. 
O.  Koopman  while  working  for  the  U.S.  Navy  during  WWII.  Koopman  called  this 
measure  of  detectability  effective  sweep  width.  The  idea  is  that  since  no  sensor  is  perfect, 
a  detection  experiment  must  consider  all  detection  opportunities  to  establish  how 
“detectable”  a  specific  target  is,  with  a  given  sensor,  within  a  given  environment. 
Koopman’s  methodology  was  incorporated  in  the  first  edition  of  the  U.S.  National  Search 
and  Rescue  Manual  in  1959,  became  accepted  by  SAR  organizations  worldwide  shortly 
afterwards,  and  has  been  included  ever  since. 

In  1978,  using  Koopman’s  methodology,  the  USCG  research  and  development 
center  ran  extensive  experiments  to  gather  more  accurate  and  realistic  SAR  data  in  actual 
search  scenarios.  The  data  collected  from  these  efforts  form  the  basis  of  the  National 
SAR  Manuals/Supplements  used  today. 

7.  Variable  Search  Speeds 

In  their  conclusion,  the  Italian  group  found  that  a  ship’s  maximum  speed  does 
affect  the  ship’s  perfonnance  in  conducting  SAR,  so  it  is  clear  that  ShipMax  should  be 
examined  in  the  NPSS  model.  What  may  be  unclear  is  whether  or  not  varying  a  searching 
entity’s  search  speed  will  have  any  effect  on  the  perfonnance  of  SAR.  Although  it  may 
be  intuitive  that  traveling  at  a  larger  speed  allows  for  more  area  to  be  searched,  this  does 
not  necessarily  mean  the  cumulative  probability  of  detection  increases.  Table  4,  which  is 
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from  the  National  Search  and  Rescue  Supplement,  indicates  that  as  search  speed 
increases,  the  correction  factor  decreases.  This  decreases  the  estimate  for  the  search 
entity’s  corrected  sweep  width,  and  in  turn  can  mean  a  lower  cumulative  probability  of 
detection.  In  other  words,  there  may  be  a  knee  in  the  curve  between  the  cumulative 


probability  of  detection  and  the  entity  search  speed. 


Search  Object 

Fixed  Wing  Speed  (Knots) 

Helicopter  Speed  (Knots) 

150  or  less 

180 

210 

60  or  less 

90 

120 

140 

Person  in  Water 

1.2 

1.0 

0.9 

1.5 

1.0 

0.8 

0.7 

Raft  - 1-4  Man 

1.1 

1.0 

0.9 

1.3 

1.0 

0.9 

0.8 

Raft  -  6-25  Man 

1.1 

1.0 

0.9 

1.2 

1.0 

0.9 

0.8 

Power  Boat  -  to  25  ft 

1.1 

1.0 

0.9 

1.2 

1.0 

0.9 

0.8 

Power  Boat  -  26-40  ft 

1.1 

1.0 

0.9 

1.1 

1.0 

0.9 

0.9 

Power  Boat  -  41-65  ft 

1.1 

1.0 

1.0 

1.1 

1.0 

0.9 

0.9 

Power  Boat  -  66-90  ft 

1.1 

1.0 

1.0 

1.1 

1.0 

1.0 

0.9 

Sail  Boat  -  to  26  ft 

1.1 

1.0 

0.9 

1.2 

1.0 

0.9 

0.9 

Sail  Boat  -  30-52  ft 

1.1 

1.0 

1.0 

1.1 

1.0 

0.9 

0.9 

Sail  Boat  -  65-90  ft 

1.1 

1.0 

1.0 

1.1 

1.0 

1.0 

0.9 

Ship  -  over  90  ft 

1.0 

1.0 

1.0 

1.1 

1.0 

1.0 

0.9 

Table  4.  Search  aircraft  speed  correction  (From  National  Search  and  Rescue 

Committee,  2000) 


An  issue  that  arises  with  the  incorporation  of  both  a  maximum  speed  factor  and  a 
separate  entity  search  speed  factor  is  that  it  becomes  possible  for  the  ship  search  speed  to 
exceed  the  ship’s  maximum  speed  if  the  two  factors  are  varied  independently  and  their 
ranges  of  potential  values  overlap.  For  this  reason,  a  control  needs  to  be  put  in  place,  not 
only  for  the  ship,  but  also  for  each  helicopter  and  UAV  as  well. 

One  method  that  can  be  applied  is  to  run  the  model  as  it  is,  and  to  run  a  simple 
check  to  verify  that  each  entity’s  maximum  speed  exceeds  or  meets  each  entity’s 
respective  search  speed.  If  it  does  not  hold  true,  the  run  can  be  labeled  “infeasible”.  The 
benefit  of  this  method  would  be  that  no  coding  would  be  required  upfront,  and  the 
analysis  process  would  have  more  clarity. 

Another  method  that  can  be  applied  for  control  is  to  convert  the  search  speed  of 
the  entity  into  a  function  of  the  maximum  speed  of  the  entity,  and  create  a  fractional 
factor  to  represent  the  entity’s  Search  Speed  Ratio  that  can  be  varied  as  an  input  as  shown 
in  equation  1 1 . 
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ShipSearchSpeed  =  ShipMaxSpeed  ■  SSSRatio 


(Equation  11) 


Ranges  (knots,  continuous): 

Ship  Max  Speed  Ship  Search  Speed  SSS  Ratio 

[0, 60]  [0,  Ship  Max  Speed]  [0,1] 

Similar  methodology  can  be  applied  to  Helicopter  and  UAV  speeds,  as  shown  in 
equations  12  and  13. 

HeloSearchSpeed  =  HeloMaxSpeed  ■  HeloSSRatio  (Pquaqon  \2) 
Ranges  (knots,  continuous): 

Helo  Max  Speed  Helo  Search  Speed  HeloSS  Ratio 

[60,160]  [60,  Helo  Max  Speed]  [0,1] 

UA  VSearchSpeed  =  UA  VMaxSpeed  ■  UA  VSSRatio  ( Equation  1 3 ) 

Ranges(knots,  continuous): 

UAV  Max  Speed  UAV  Search  Speed  UAVSS  Ratio 

[150,230]  [150,  UAV  Max  Speed]  [0,1] 

The  benefit  of  this  method  is  that  only  feasible  computations  are  conducted, 
thereby  reducing  the  amount  of  computation  as  well  as  the  time  required  for  computation. 
The  latter  method  seems  more  efficient  than  the  former,  so  it  is  incorporated  in  the  NPSS 
model  by  requiring  the  user  to  enter  search  speed  ratios,  rather  than  actual  search  speeds, 
to  obtain  the  results. 

8.  Search  Box  Generation 

In  order  to  analyze  the  same  scenario  introduced  in  the  Italian  model,  the 
rectangular  body  of  water  in  which  the  SAR  mission  is  to  be  conducted,  can  be  generated 
in  the  same  fashion. 

The  SAR  OPV  begins  the  search  an  initial  distance  in  nautical  miles,  labeled 
DatumCDR,  measured  from  the  ship’s  location  to  the  center  of  the  datum  of  uncertainty. 
This  datum  of  uncertainty  represents  the  ship’s  last  known  coordinates,  and  has  a  radius 
measured  in  nautical  miles  associated  with  it  of  length  DatumU.  With  the  same 
assumptions  as  in  the  Italian  model,  the  target’s  constant  speed  is  equal  to: 
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DriftSpeed  =  —  •  WindSpeed 

2-0  (Equation  14) 

As  in  the  Italian  model,  the  direction  of  the  target’s  movement  is  assumed  to  be 

71 

bounded  by  a  fixed  value  equal  to  —  radians(labeled  as  WindBounds),  centered  in  the 

8 

direction  of  the  wind  labeled  WindD  (in  radians). 

Using  the  last  known  location  of  the  datum,  and  the  wind  velocity  (speed  and 
direction),  the  rectangular  search  box  is  constructed  through  a  similar  methodology  used 
by  the  Italian  model  shown  in  Figure  3.  Figure  6  shows  the  inputs  necessary  to  generate 
the  search  box.  The  inputs  are  shown  in  green,  and  WindBounds  is  in  red  since  it  is  a 
fixed  value  throughout  all  excursions  of  the  model.  The  search  area  A  now  constructed 
will  be  known  in  a  given  scenario  with  units  of  nm: . 


— - 

ShipMax 

1 

i 

1 

_ _ 

— 

|  Datum  U 

— 1 

|  Search  Area 

- 

/ 

\ 

WindSR 

1 

1 

WindD 

Datum  CDR 

i 

Figure  6.  Inputs  in  search  area  generation 


9.  Wind  Speed  Correction 

Incorporating  Wind  Speed  into  the  NPSS  model  study  requires  some  careful 
considerations.  Although  ideally  it  would  make  sense  to  allow  the  Wind  Speed  to 
confonn  to  typical  wind  ranges,  it  would  allow  for  certain  unusual  situations  to  arise.  For 
example,  consider  a  scenario  when  the  ship  is  searching  for  the  target  without  a 
helicopter  or  UAV  assets,  and  the  ShipMax  is  relatively  slow.  It  would  be  possible  for 
the  Target’s  Drift  Speed  to  exceed  ShipMax,  which  would  create  a  situation  in  which  the 
calculation  of  the  MOE  would  become  infeasible.  Because  the  Target’s  Drift  Speed  is  a 
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direct  function  of  the  Wind  Speed,  it  would  be  ideal  to  limit  the  Wind  Speed  such  that  the 
Target’s  Drift  Speed  never  exceeds  the  maximum  speed  of  the  ship  in  a  given  scenario.  In 
order  to  accomplish  this,  a  new  factor  called  the  Wind  Speed  Ratio  (WindSR)  can  be 
created,  similar  to  the  factor  created  in  the  issue  with  ShipMax  and  the  Ship  Search 
Speed.  Note,  that  only  the  Ship  speed  would  ever  affect  this  issue;  since  the  helicopter 
and  the  UAV  have  60  knots  and  150  knots  respectively  as  their  lower  bound  search  speed 
limits,  which  would  never  dip  below  the  maximum  possible  Wind  Speed  of  60  knots. 


WindSpeed  <  •  SSS]  ■  WindSR 


(Equation  15) 


Using  substitution  of  equation  1 1  into  equation  15: 

2.0 

WindSpeed  <  [ - ShipMax  ■  SSSRatio]  ■  WindSR 

1-5  (Equation  16) 

Ranges(knots,  continuous): 

ShipMax  Wind  Speed  WindSR 

[10,60]  [0,  1.332*ShipMax*WindSR]  [0,0.999] 


10.  Coverage  Factor 

Now  that  the  Corrected  Sweep  Widths  of  each  search  asset,  the  size  of  the  search 
area,  and  the  search  speeds  for  a  given  scenario  can  all  be  calculated,  these  values  can  be 
used  to  construct  the  coverage  factor.  The  coverage  factor  represents  the  search  effort 
expended  in  the  numerator  and  total  search  area  in  the  denominator,  using  the  following 
fonnula: 

CoverageF actor  =  — — 

SearchArea  (Equation  17) 

where  v  represents  the  search  entity’s  search  speed.  The  estimation  and  calculation  of  the 

sweep  width,  the  velocities  of  each  search  entity,  and  the  search  area  have  already  been 

discussed;  the  only  question  with  this  new  term  is  how  time  will  be  incorporated  in  the 

model.  One  method  would  be  to  incorporate  a  while-loop,  to  have  time  count  forward 

until  the  measures  of  effectiveness  are  met.  A  simpler  approach  would  be  to  increment 

time  forward  a  fixed  amount  of  time,  and  to  adjust  as  necessary,  as  the  measure  of 
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effectiveness  requires  additional  time,  and  at  each  increment  of  time  to  calculate  each 
search  entity’s  Coverage  Factor  given  their  respective  sweep  width,  search  velocity,  and 
the  design  point’s  search  area.  The  NPSS  model  uses  the  latter  approach  and  has  an 
increment  equal  to  5%  of  an  hour.  Additionally,  the  search  activity  is  cut  off  at  a 
maximum  of  65  hours.  This  places  a  limitation  on  the  model;  however  as  far  as  SAR 
operations  are  concerned,  it  seems  to  be  a  reasonable  length  of  time  for  missions  to  be 
conducted. 

11.  Search  Pattern  and  Detection  Probability 

Figure  7  shows  the  search  pattern  for  the  Italian  model.  The  blue  path  lays  out  the 
sweeping  pattern  of  the  OPV,  and  the  red  path  lays  out  the  sweeping  pattern  of  the 
helicopter  with  Wu  representing  the  visual  sweep  width  of  the  helicopter  and  W°P}  u 
representing  the  visual  sweep  widths  of  the  helicopter  and  OPV  combined  in  the  Italian 
model. 


Figure  7.  Italian  model  search  pattern  (From  Anghinolfi  et  ah,  2011) 

An  alternative  to  the  specific  pattern  contained  in  the  Italian  Model  is  to  generally 
consider  that  the  search  platforms  search  the  area  by  a  series  of  parallel  sweeps,  and 
apply  a  result  from  Search  Theory  known  as  the  Inverse  Cube  Law  (ICL)  of  Detection 
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(Wagner,  Sanders,  &  Mylander,  1999)  to  obtain  a  good  approximation  of  probability  of 
detection  for  area  search  based  on  coverage  factor. 

From  Wagner,  the  ICL  result  is  that  probability  of  detection  for  this  area  search 
can  be  estimated  using  the  Standard  Normal  Distribution  as  follows: 


z 

Pd  =  2  •  J  </>(t)dt 


0  (Equation  1 8) 

where  (j)  is  the  standard  normal  probability  density  function  with  mean  zero  and  variance 
one,  and 


z  = . 


■  Cover  ageF 'actor 


2  (Equation  19) 

The  ICL  comes  from  the  idea  of  conducting  a  parallel  search  with  multiple 
sweeps  of  the  area  over  parallel  paths  as  shown  in  Figure  8. 


1  ^  1  -  — T*^ 

12  3  4 


Figure  8.  Four  parallel  searchers  (From  Washburn,  2002) 

The  parallel  sweeps  could  be  by  multiple  search  entities,  or  by  a  single  search 
entity  that  repositions  at  the  end  of  each  sweep  to  conduct  the  next  parallel  sweep.  The 
area  under  each  curve  in  Figure  8  represents  the  search  entity’s  sweep  width.  The  order 
the  tracks  are  traversed  does  not  matter.  Using  the  ICL,  “edge  effects”  are  not  considered. 
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The  ICL  assumes  that  the  Target  is  uniformly  randomly  distributed  in  A  (i.e., 
likely  to  be  found  in  one  part  of  the  region  as  in  any  other),  the  search  conducts  a  parallel 
sweep  pattern,  and  each  sweep  of  a  given  search  entity  is  independent  of  each  of  the  other 
of  that  particular  search  entity.  The  unifonnly  randomly  distributed  target  assumption  is 
also  used  in  the  Italian  model. 

This  Pd  computed  with  the  ICL  represents  the  probability  that  the  area  search 
results  in  at  least  one  detection  ,  which  is  a  cumulative  probability  of  detection  based  on 
coverage  factor  by  a  particular  search  asset  (with  sweep  width  W,  search  speed  v,  and 
search  time  t).  With  the  ICL,  when  coverage  factor  =  1,  Equation  19  shows  that 


z  = 


1.253  ,  and  Equation  18  provides  the  result  that  Pd  is  0.789.  The  time  to 


achieve  coverage  factor  of  1  can  be  calculated  from  Equation  17.  Setting  coverage 
factor  to  1  gives: 

SearchArea 
t  = - 

W ' v  (Equation  20) 

The  time  that  is  calculated  in  Equation  20  can  now  be  known  as  a  “pass”. 
Although  it  is  not  necessarily  the  time  required  to  achieve  complete  coverage  of  the 
search  area  (this  would  be  the  case  if  the  search  entity  utilized  a  cookie-cutter  sweep 
width),  it  will  be  used  to  represent  a  uniform  increment  of  time.  Once  the  coverage  factor 
has  reached  a  value  of  1,  the  maximum  CDP  of  0.789  is  achieved  for  that  search  entity, 
while  other  search  entities  are  allowed  to  continue  searching  and  achieving  greater 
respective  CDPs.  This  is  unrealistic,  because  it  means  the  faster  search  assets  (helicopter 
or  UAV)  might  spend  much  less  time  on  the  search  operation  than  the  ship.  A  simple  fix 
to  this  issue  is  to  begin  another  independent  search  of  the  search  box  each  time  a  search 
entity  other  than  the  ship  completes  searching  the  search  box;  every  time  a  search  entity 
completes  a  “pass”. 

Although  this  fix  solves  one  problem,  it  brings  up  another  interesting  issue;  given 
numerous  design  configurations,  in  a  worst  case  scenario  what  would  be  the  maximum 
number  of  passes?  Would  the  NPSS  model  be  able  to  handle  this?  It  turns  out,  that  only  a 
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small  number  of  passes  per  entity  really  need  to  be  accounted  for,  since  the  CDPs  of  a 
single  search  entity  rapidly  approaches  1  as  the  number  of  passes  increase. 

After  a  single  pass  once  the  coverage  factor  reaches  1,  z  is  roughly  equal  to  1.253, 
and  the  CDP  maxes  out  at  roughly  0.789.  After  the  second  pass  is  completed,  again 
assuming  independence  between  the  two  passes  of  the  same  search  entity,  the  CDP  is: 

Pd(after  second  pass)  =  1  -  (1  -  Pd\\stPass ))  •  (1  -  Pd(2ndPass )) 

(Equation  21) 

=0.955 

The  same  analogy  can  be  applied  to  three,  four  and  five  passes,  resulting  in  CDPs: 

Pd  after  third  pass  =  0.990 

Pd  after  fourth  pass  =  0.998 

Pd  after  fifth  pass  =  0.999 

Since  the  NPSS  model  will  always  have  at  least  one  search  entity  searching  for  a 
target,  it  is  safe  to  assume  that  anything  beyond  five  passes,  would  add  negligible  CDP, 
and  therefore  the  NPSS  model  will  only  be  expected  to  calculate  CDPs  for  each  search 
entity  with  a  total  of  up  to  five  passes  of  the  search  box  per  entity.  Because  the 
assumptions  required  for  the  ICL  are  satisfied,  the  NPSS  model  can  incorporate  using  the 
ICL  to  calculate  CDP, 

12.  Number  of  Helicopters/UAVS 

In  order  to  incorporate  the  effects  of  multiple  helicopters  and  UAVs  into  the 
model,  the  assumption  can  be  made  that  all  of  the  search  entities  are  independent  of  one 
another.  Again  this  assumption  can  be  made  since  no  new  infonnation  will  be  passed  to 
each  of  the  search  entities  up  until  if  the  target  is  found,  and  the  objective  of  the  NPSS 
model  will  be  to  evaluate  the  performance  of  the  configuration  of  the  OPV  in  the  SAR 
mission. 

Once  the  Pd  of  a  single  search  entity  has  been  computed  using  the  ICL,  having 
independence  across  each  search  entity  we  may  use  De  Morgan’s  laws  to  conduct  a 
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calculation  for  total  Pd  using  a  specified  number  of  helicopters  labeled  NumHelo,  and  a 
specified  number  of  UAVs  labeled  NumUAV. 

Pd(two  helicopters) 

=  1  -  [(1  -  Pd  {Ship))  ■  (1  -  Pd{\stHelo))  ■  (1  -  Pd  (2ndHelo))\  (Equation  22) 

Since  Pd{\stHelo)  =  Pd(2ndHelo) 

Pd(N  helicopters)  =  1  -  [(1  -  Pd  {Ship))  ■  (1  -  Pd{\stHelo))A  ]  (Equation  23) 

(Similar  method  applies  for  UAVs) 

Pd(N  helicopters,  M  UAVs) 

=  1  -  [(1  -  Pd{Ship))  •  (1  -  Pd{Helo)f  •  (1  -  Pd{UA  V))M  ]  (Equation  24) 

13.  DOE-NOLH 

Up  to  this  point,  the  model  has  accumulated  a  total  of  fifteen  different  factors  to 
take  into  account  (shown  in  Table  5).  ShipMax,  Datum  U,  WindSR,  WindD,  and 
DatumCDR  are  used  in  developing  the  search  box.  Search  speeds,  the  visibility,  the  target 
type,  and  ECF  affect  each  searching  entity’s  respective  sweep  width. 

The  idea  behind  the  Design  of  Experiments  (DOE)  process  is  to  systemically 
structure  the  values  of  multiple  input  factors  in  an  attempt  to  gain  insights  about  how  the 
input  factors  and  their  interactions  affect  the  response  (Sanchez,  Lucas,  Sanchez,  Wan,  & 
Nannini,  2012).  Using  a  DOE  approach  is  a  very  efficient  way  to  explore  the  SAR 
scenario,  and  much  more  efficient  than  a  trial  and  error  approach. 

The  DOE  design  which  uses  465  design  points  has  been  custom  generated  by 
Alex  Maccahnan  (MacCahnan,  Vieira,  &  Lucas,  2012)  with  correlations  between  the 
columns  in  the  design  matrix  within  the  interval  (-0.05,  0.05).  It  uses  all  15  factors;  11 
continuous,  3  discrete  (two  with  3  levels,  and  one  with  25  levels),  and  1  categorical  factor 
(with  4  levels),  with  the  low  and  high  level  of  the  range  of  the  factors  indicated  in  Table 
5.  Note  that  the  target  is  treated  as  discrete,  rather  than  categorical,  since  the  Target  factor 
(1-25)  corresponds  to  a  general  increase  in  target  size,  and  a  25-level  categorical  factor 
would  require  a  substantially  larger  design.  Note  that  for  continuous  factors,  the  number 
of  levels  using  a  space  filling  nearly  orthogonal  Latin  hypercube  (NOLH)  design 
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corresponds  to  465.  “Nearly  orthogonal”  refers  to  a  good  space-filling  design  in  which 
the  design  points  are  scattered  throughout  the  experimental  region  with  minimal 
unsampled  regions,  and  are  nearly  orthogonal;  for  example,  all  correlations  between  the 
columns  in  the  design  matrix  are  in  the  interval  (-0.03,  0.03)  (Cioppa  &  Lucas,  2007).  If 
the  factors  for  the  model  were  calculated  using  a  full  factorial  design  with  the  number  of 
levels  shown  in  Table  5,  the  number  of  design  points  (calculated  below  using  equation 
25)  would  be  astronomical. 

#  of  Design  Points  (full  factorial)  =  (#  of  Levels)(#  of  factors)  (Equation  25) 

=  (465)(11)*(4)(1)*(3)(2)*(25)(1>  =  197,799,409,721,224,801,632,875,976,562,500 
Even  if  the  continuous  factors  were  explored  at  a  much  smaller  number  of  levels 
(say,  1 1),  a  full  factorial  would  still  result  in  over  2.5xl014  design  points. 


Factor  Type 

Input  Factor 

Description 

it  of  Levels 

Lower  Bound 

Upper  Bound 

Units 

Decision 

ShipMax 

Maximum  Ship  Speed 

465 

10 

60 

knots 

Decision 

NumHelo 

Number  of  Helos  in  the  scenario 

3 

0 

2 

helicopters 

Decision 

NumUAV 

Number  of  UAVs  in  the  scenario 

3 

0 

2 

UAVs 

Noise 

DatumCDR 

Initial  Distance  of  ship  to  center  of  datum 

465 

50 

200 

nm 

Noise 

UAVMax 

Maximum  UAV  Speed 

465 

150 

230 

knots 

Noise 

HeloMax 

Maximum  Helo  Speed 

465 

60 

160 

knots 

Noise 

ECF 

Environmental  Control  Factor(Wind,  Sea  state  thresholds) 

4 

1 

4 

Noise 

Vis 

Visibility 

465 

1 

30 

nm 

Noise 

UAVSS(Fract  of  Max) 

UAV  Search  Speed(Represented  as  a  fraction  of  the  Max  UAV  Speed) 

465 

0 

1 

Noise 

ShipSS(Fract  of  Max) 

Ship  Search  Speed(Represented  as  a  fraction  of  the  Max  Ship  Speed) 

465 

0 

1 

Noise 

HeloSS(Fract  of  Max) 

Helo  Search  Speed(Represented  as  a  fraction  of  the  Max  Helo  Speed) 

465 

0 

1 

Noise 

WindSR 

Wind  Speed(Represented  as  a  fraction  of  the  Max  Ship  Speed) 

465 

0 

1 

Noise 

WindD 

Wind  Direction(in  radians) 

465 

0 

6.283185307 

radians 

Noise 

Datumll 

Datum  Uncertainty 

465 

0 

10 

nm 

Noise 

Target 

Type  of  Search  Target 

25 

1 

25 

Table  5.  Input  factor  ranges  and  levels 


14.  Measure  of  Effectiveness 

In  any  kind  of  disaster,  SAR  operations  are  vital,  and  time  is  of  the  essence. 
Although  the  cumulative  probability  of  detection  is  important,  it  is  also  important  to 
minimize  search  time  in  an  attempt  to  rescue  the  survivor(s)  if  he/she  (they)  are  still  alive. 

With  two  competing  objectives,  it  becomes  difficult  to  localize  what  the  proper 
objective  should  be  to  use  as  a  measure  of  effectiveness.  One  method  in  dealing  with  this 
issue  is  to  make  the  measure  of  effectiveness  a  function  of  the  competing  objectives,  and 
then  using  a  multi-pronged  approach,  analyze  various  different  ratios  of  the  objectives. 
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In  this  case,  instead  of  solely  using  a  given  ship  design  point’s  CDP  as  the 
measure  of  effectiveness,  or  the  average  time  required  to  locate  the  target,  the  CDP  can 
be  tabularized  as  a  function  of  time.  Allowing  this  would  enable  the  analysis  to  be 
conducted  for  any  given  threshold  of  time  as  a  cutoff.  For  simplicity,  the  measure  of 
effectiveness  “Average  Time  Required  given  a  Pd  threshold”  can  be  sliced  into  four 
different  bins: 


MOE  1 :  Time  Required  for  Configuration  to  Achieve  0.95  Pd 
MOE  2:  Time  Required  for  Configuration  to  Achieve  0.75  Pd 
MOE  3:  Time  Required  for  Configuration  to  Achieve  0.50  Pd 
MOE  4:  Time  Required  for  Configuration  to  Achieve  0.25  Pd 
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III.  ANALYSIS 


For  the  analysis  of  the  search  model’s  output,  three  analysis  objectives  need  to  be 
completed.  Firstly,  the  NPSS  model’s  data  can  be  represented  in  a  statistical  metamodel 
built  using  the  “full”  set  of  factors  varied  during  the  experiment  (The  term  “metamodel” 
is  used  to  avoid  confusion  between  the  NPSS  model  itself,  and  the  analysis  regression 
models).  Using  this  analysis,  it  may  be  useful  to  break  down  and  understand  under  which 
circumstances  the  decision  factors  have  the  most  impact  on  the  performance  of  the  ship  in 
its  mission.  In  this  section,  the  name  of  each  metamodel  is  composed  of  the  MOE 
followed  by  an  “a”  to  represent  that  all  factors  (noise  and  control)  are  incorporated  in  the 
model. 

Secondly,  the  NPSS  model’s  data  can  be  used  to  build  a  statistical  metamodel 
using  only  a  handful  of  the  total  number  of  factors  in  the  NPSS  model,  specifically,  using 
the  operational  “decision”  factors.  The  purpose  of  this  objective  is  to  link  the  importance 
of  each  factor  amongst  the  group  of  controllable  factors,  to  operational  performance  of 
the  NPSS  mission.  This  is  useful  because  it  provides  an  idea  of  how  physically  different 
ship  configurations  directly  impact  the  measure  of  effectiveness  in  the  presence  of 
numerous  noise  variables.  In  this  section,  the  name  of  each  metamodel  is  composed  of 
the  MOE  followed  by  a  “b”  to  represent  that  the  noise  factors  are  not  incorporated  in  the 
model. 

Lastly,  the  NPSS  model’s  data  can  be  used  in  an  analysis  of  how  the  results  of 
this  model  compare  to  the  Italian  group’s  results.  The  first  and  second  analysis  objectives 
are  conducted  on  all  four  measures  of  effectiveness. 

The  tool  that  will  be  used  primarily  in  this  analysis  is  the  JMP  software  package. 
JMP  is  a  computer  program  first  developed  by  John  Sail,  which  allows  complex 
statistical  analyses.  The  benefits  of  the  JMP  software  package  include  ease  of  use,  and  the 
dynamic  behavior  of  its  graphics,  data  tables,  which  facilitate  the  understanding  of  even 
the  most  complex  problems  (Jones  &  Sail,  2011). 
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A 


ANALYSIS  1:  THE  FULL  FACTOR  METAMODEL 


The  first  thing  that  would  be  interesting  to  look  at  is  to  get  an  idea  of  what  the 
distribution  of  the  MOE  outputs  look  like.  Figure  9  shows  the  distribution  summary  of  all 
four  MOEs.  Immediately  what  is  apparent  is  the  large  density  of  counts  in  each  of  the  60- 
70  count  bins  in  the  figure  (circled  in  red). 
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Figure  9.  Distributions  of  MOEs  -  Analysis  1 


These  spike  points  are  obviously  due  to  the  65  hour  limit  placed  on  the  NPSS 
model’s  coverage  factor  time  limit.  The  outputs  show  no  signs  of  having  negative  values, 
which  is  expected  since  the  time  output  cannot  be  negative  in  this  model.  The  cumulative 
probability  threshold  increases  from  0.25  to  0.95,  the  mean  and  median  values  of  the 
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MOEs  increase  in  time,  which  is  intuitive,  and  function  as  expected.  Additionally  the 
outputs  have  large  ranges,  so  it  makes  sense  to  do  further  analyses  to  see  which  factors 
affect  the  outputs 


1.  Analysis  1  Rough  Regression 

To  get  a  rough  idea  of  which  factors  in  the  full  factor  model  are  important  to  each 
of  the  MOEs,  each  set  of  data  can  be  fit  to  a  2nd  order  regression  using  a  forward 
Stepwise  approach  with  a  minimum  Bayesian  information  criterion  (BIC)  stopping 
condition.  In  these  regressions,  the  metamodels  are  able  to  achieve  coefficient  of 
determination  (Rsquare)  adjusted  value  of  0.767,  0.705,  0.674,  and  0.613  for  the  MOEs 
with  Pd  thresholds  of  0.95,  0.75,  0.5,  and  0.25  respectively.  The  regression  summaries 
and  the  first  few  significant  factors  for  each  of  the  MOE  metamodels  are  shown  in 
Figures  10-17. 
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Figure  10.  Regression  summary  (MOE  la) 
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a  Sorted  Parameter  Estimates 
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Figure  11.  First  few  significant  factors  (MOE  la) 
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Regression  summary  (MOE  2a) 
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6  Sorted  Parameter  Estimates 
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Figure  13.  First  few  significant  factors  (MOE  2a) 
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Figure  14.  Regression  summary  (MOE  3a) 
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a  Sorted  Parameter  Estimates 
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Figure  15.  First  few  significant  factors  (MOE  3a) 
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Figure  16.  Regression  summary  (MOE  4a) 
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a  Sorted  Parameter  Estimates 
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Figure  17.  First  few  significant  factors  (MOE  4a) 


The  Rsquare  adjusted  values  in  each  of  the  models  indicate  that  these  particular 
metamodels  are  able  to  explain  61.3-76.7%  of  the  variation  in  each  respective  model 
using  2nd  order  polynomials.  Looking  at  the  actual  versus  predicted  plots  indicate  that 
the  regression  metamodels  have  a  difficult  time  dealing  with  low  values  of  the  MOE, 
where  they  predict  search  times  below  zero,  as  well  as  behavior  around  time  equal  to  65 
hours,  where  the  model  has  a  limitation;  these  problem  areas  are  circled  in  red  on  Figures 
10,  12,  14,  and  16.  Due  to  these  issues,  along  with  the  overly  large  number  of  terms  in  the 
metamodels  themselves  (between  28  and  36),  it  may  be  a  better  idea  to  try  to  fit  a 
partition  tree  to  the  data;  the  partition  tree  metamodel  may  be  able  to  better  account  for 
instantaneous  jumps  in  the  data. 

2.  Partition  Tree  Analysis 

After  the  first  few  splits  in  the  each  of  the  four  MOEs  fitted  by  partition  trees 
using  all  factors  in  the  model,  the  Rsquares  of  the  partition  trees  begin  to  experience 
diminishing  returns  as  more  splits  occur.  Rsquares  of  0.519  0.375,  0.537,  0.356  are 
achieved  after  6,  5,  11,  11  splits  for  MOEs  la,  2a,  3a,  and  4a  respectively  (Figures  18,  21, 
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24,  28).  Taking  a  look  at  factor  contributions  of  each  of  these  metamodels  yield  that  some 
factors  up  to  its  respective  split  contribute  nothing  to  the  sum  of  squares  (shown  in 
Figures  20,  23,  27,  31). 

a.  MOE  la:  Average  Time  to  Achieve  Cumulative  Pd  of  0. 95 

Comparing  Figures  11  and  19,  it  is  interesting  to  note  the  biggest 
contributors  to  the  full  factor  regression  and  partition  metamodels  seem  to  match  up 
relatively  well.  From  all  potential  factors  (including  noise),  the  partition  tree  emphasizes 
the  importance  of  the  presence  (at  least  one)  of  a  UAV  for  MOE  1  a  as  the  most  important 
factor.  It  is  interesting  that  in  the  case  that  a  ship  lacks  the  presence  of  a  UAV,  the 
metamodel  suggests  that  a  search  helicopter  would  be  the  next  best  factor.  Additionally, 
in  the  case  that  a  ship  configuration  lacks  both  helicopters  and  UAVs,  according  to  the 
metamodel,  the  ship  would  benefit  the  most  for  ensuring  the  ship’s  speed  has  the 
capability  to  move  quicker  than  27  knots. 


i  ■»  Partition  for  0.95  2 


Split  1  [  Prune 


RSquai  e 

RMSE 

N 

Numbei 
of  Splits 

Imputes 

AlCc 

0.519 

18.288843 

484 

6 

53 

4203.13 

Figure  18.  Partition  summary  (MOE  la) 
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Figure  19.  Partition  tree  (MOE  la) 
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Figure  20.  Contributing  factors  (MOE  l.a) 


b.  MOE  2a:  Average  Time  to  Achieve  Cumulative  Pd  of  0. 75 

Similar  to  the  comparison  of  MOE  la’s  contributing  factors,  MOE  2a’ s 
regression  and  partition  tree  contributing  factors  can  be  compared  (Figures  13  and  22). 
WindSR,  Target,  and  NumUAV  are  still  contributing  factors  in  the  partition  model, 
however  ShipMax  has  become  less  important,  and  NumHelo  and  WindD  have  become 
more  important  in  the  partition  metamodel.  Also,  in  the  left  branch  of  the  partition 
metamodel  (Figure  22),  the  factor  Target  has  been  split  at  2.  This  does  make  some  sense, 
since  the  target  data  comes  from  the  National  SAR  manual,  specifically  the  uncorrected 

37 


sweep  width  tables  (Table  2).  Looking  at  these  tables,  it  is  apparent  that  each  target’s 
uncorrected  sweep  width  increases  as  a  function  of  visibility,  except  for  Target  1,  namely 
“Person  in  Water”.  This  may  explain  the  natural  split  noted  in  the  partition  tree. 

In  the  right  branch  of  the  partition  tree,  the  WindSR  is  relatively  high, 
meaning  the  actual  wind  speed  in  the  design  point  is  high  as  well.  Since  the  drift  speed  of 
the  target  is  a  function  of  wind  speed,  and  the  search  area  is  a  function  of  drift  speed,  it 
may  imply  that  everything  in  the  right  branch  tends  to  occur  in  a  larger  search  area  A.  The 
next  split  in  the  right  branch  occurs  in  the  WindD,  the  wind  direction.  Interestingly 
enough,  it  splits  at  the  radian  value  3.126,  which  is  roughly  the  value  of  n.  Along  with  the 
wind’s  direction,  the  WindD  also  represents  the  general  drifting  direction  of  the  target 
(bounded  by  Windbounds).  In  essence,  the  split  on  WindD  emphasizes  which  side  above 
or  below  the  center  of  the  initial  datum  location  the  search  area  resides.  If  WindD  e 
(n,2n),  the  search  area  will  most  likely  reside  below  the  center  of  the  initial  target  datum, 
meaning  the  ship  generally  has  less  distance  to  travel  to  reach  the  search  area.  If  the 
WindD  e  (0,n),  the  search  area  will  most  likely  reside  above  the  center  of  the  initial  target 
datum,  meaning  the  ship  generally  has  more  distance  to  travel  to  reach  the  search  area.  In 
the  specific  case  that  the  search  area  is  closer  to  the  ship,  in  the  presence  of  relatively 
high  winds,  the  presence  of  a  UAV  is  the  most  important  factor  over  every  other  factor  in 
the  metamodel,  followed  by  the  presence  of  a  helicopter  when  lacking  the  presence  of  a 
UAV. 
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Figure  2 1 .  Partition  summary  (MOE  2a) 


Figure  22.  Partition  tree  (MOE  2a) 


39 


Column  Contributions 

Number 

Tenn 

of  Splits 

ss 

ShipMax 

0 

0.000 

UAVMax 

0 

0.000 

HeloMax 

0 

0.000 

WindD 

1 

26424.715 

1 

Vis 

0 

0.000 

UAVSS(FractofMax) 

0 

0.000 

HeloSS(FractofMax) 

0 

0.000 

ShipSS(Fract  of  Max) 

0 

0.000 

WindSR 

1 

33676.331 

DatumU 

0 

0.000 

DatumCDR 

0 

0.000 

NumUAV 

1 

5860.112 

1  ( 

NumHelo 

1 

10162.445 

Target 

1 

18210.870 

1 

ECF 

0 

0.000 

Figure  23.  Contributing  factors  (MOE  2a) 


c.  MOE  3a:  Average  Time  to  Achieve  Cumulative  Pd  of  0.5 


Comparing  the  regression  and  partition  metamodels  of  MOE  3  a  (Figure  15 
with  Figures  25  and  26),  the  factors  Target,  NumUAV,  and  NumHelo  can  be  noted  to 
have  become  more  important  in  the  partition  tree,  otherwise  every  other  factor  seems  to 
be  as  important  in  both  metamodels  relative  to  the  other  factors. 
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Figure  24.  Partition  summary  (MOE  3  a) 


Looking  at  the  left  branch  of  the  partition  metamodel  (Figure  26),  the 
initial  important  break  occurs  in  the  factor  Target,  again  at  the  value  of  2,  and  may  be 
caused  by  the  same  issue  discussed  previously  in  MOE  2a.  In  the  partition  metamodel, 
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assuming  the  Target  factor  is  any  target  between  the  2  and  25,  and  that  the  WindS R  is 
relatively  small,  NumUAV  is  the  most  important  contributing  factor.  Lacking  the 
presence  of  a  UAV,  the  presence  of  a  helicopter  then  becomes  the  most  important  factor. 


Figure  25.  Partition  tree  (MOE  3a  -  left  branch) 


Looking  at  the  right  branch  of  the  partition  tree  metamodel(Figure  26), 
after  assuming  the  Target  is  not  a  single  individual  in  the  water,  WindSR  is  relatively 
high,  and  the  wind  is  blowing  towards  the  ship  (WindD  e  (n,2n)),  the  presence  of  UAVs 
is  the  most  important  factor,  followed  by  the  presence  of  helicopters  when  lacking  UAVs. 
If  the  wind  is  blowing  away  from  the  ship  (WindD  e  (0,  n)),  the  most  important  factor 
becomes  whether  or  not  the  ship’s  maximum  speed  exceeds  14  knots. 
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Figure  26.  Partition  tree  (MOE  3a  -  right  branch) 


i  Column  Contributions 
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Figure  27.  Contributing  factors  (MOE  3a) 


d.  MOE  4a:  Average  Time  to  Achieve  Cumulative  Pd  of  0.25 

Comparing  MOE  4a’ s  regression  metamodel  with  its  partition  tree  (Figure 
17  with  29  and  30),  the  importance  of  the  factors  ShipMax,  Target,  and  UAVMax 
increase  in  the  partition  tree.  The  most  interesting  factor  in  this  partition  tree,  would  have 
been  the  presence  of  UAVMax,  however  since  the  split  only  has  a  count  of  5  design  point 
occurrences,  it  does  not  impact  the  overall  model  significantly. 
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Figure  28.  Partition  summary  (MOE  4a  threshold) 


Figure  29.  Partition  tree  (MOE  4a  -  left  branch) 
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Figure  30.  Partition  tree  (MOE  4a  -  right  branch) 
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Figure  3 1 .  Contributing  factors  (MOE  4a) 


e.  Comparing  all  Four  MOEs 

Collecting  the  broad  spectrum  of  insights  provided  by  each  individual 
MOE  metamodel,  a  pattern  begins  to  develop.  Looking  at  contributions  of  factors  to  their 
respective  MOE  model  (Figure  32),  the  factors  of  NumUAV  and  NumHelo  can  be  seen 
heavily  contributing  to  MOE  la,  and  becoming  less  and  less  important  as  the  CDP 
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threshold  for  the  MOE  decreases,  reaching  no  contribution  in  MOE  4a.  Alternatively,  the 
factor  Target  and  ShipMax  seemingly  contribute  very  little  to  MOE  la,  gradually 
increase  their  contributions  to  the  MOEs  as  the  CDP  threshold  for  the  MOEs  decrease, 
and  eventually  become  the  most  important  factors  in  MOE  4a.  Additionally,  it  is 
important  to  note  that  WindSR  is  a  significant  factor  in  all  four  MOEs,  and  that  HeloMax 
(the  helicopter’s  maximum  speed),  Vis,  DatumU,  EOF,  and  all  three  of  the  search  speed 
ratios  UAVSS,  HeloSS,  and  ShipSS  have  negligible  contributions  in  all  four  MOEs. 
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Figure  32.  Contributing  factors  (MOEs  la,  2a,  3a,  4a) 


3.  Number  of  Splits  versus  Rsquare 

To  get  an  idea  of  the  variability  in  the  partition  fit,  the  Rsquare  of  each  of  the  “a” 
series  MOEs  is  tabularized  as  a  function  of  CDP  threshold  and  the  number  of  splits  (see 
Table  6).  Note  that  for  each  MOE  (see  Table  6),  the  Rsquare  remains  relatively  low,  even 
after  twelve  splits,  meaning  that  it  is  relatively  difficult  to  predict  the  performance  of  the 
ship  with  the  NPSS  model  in  the  presence  of  the  various  noise  factors. 
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Pd  threshold 


0.95 

0.75 

0.5 

0.25 

1 

0.114 

0.129 

0.156 

0.193 

2 

0.327 

0.194 

0.26 

0.274 

3 

0.328 

0.293 

0.347 

0.314 

4 

0.398 

0.32 

0.437 

0.326 

S 

0.437 

0.375 

0.448 

0.35 

6 

0.438 

0.406 

0.453 

0.352 

7 

0.522 

0.414 

0.456 

0.357 

8 

0.582 

0.422 

0.456 

0.362 

9 

0.594 

0.429 

0.473 

0.363 

10 

0.607 

0.432 

0.48 

0.365 

11 

0.612 

0.436 

0.482 

0.365 

12 

0.624 

0.468 

0.482 

0.366 

Table  6.  Rsquare  for  MOE  “a”  series 


Table  6  can  also  be  used  to  decide  when  to  stop  splitting.  For  example,  for  the  Pd 
threshold  of  0.25,  there  is  very  little  improvement  in  Rsquare  between  the  fifth  and 
twelfth  splits. 

B.  ANALYSIS  2:  THE  METAMODEL  FOR  THE  CONTROLLABLE 

FACTORS 

Instead  of  using  all  of  the  SAR  model’s  factors  in  the  partition  metamodels,  a 
partition  tree  can  be  built  using  only  the  decision  factors.  In  doing  so,  it  may  allow  better 
comparisons  amongst  the  modeling  factors  than  the  previous  analysis,  at  the  cost  of  a 
decreased  ability  to  obtain  a  good  fit  in  the  absence  of  noise  factor  terms.  However,  if 
ship  configurations  can  be  found  that  yield  low  means  and  low  standard  deviations  of  the 
MOEs,  these  can  be  considered  robust  ship  configurations.  Once  again,  partition  trees  of 
each  one  of  the  four  MOEs  will  be  obtained,  and  then  compared  amongst  each  other. 
Additionally,  an  analysis  of  number  of  splits  versus  partition  metamodel  Rsquare  will  be 
conducted. 

1.  Partition  Tree  Analysis:  Comparing  MOE  lb,  2b,  3b,  4b 

In  MOE  lb’s  partition  metamodel  (Figure  34),  the  factor  with  the  most  impact  on 
the  outcome  is  NumUAV;  whether  or  not  UAVs  are  present  in  the  ship  configuration. 
This  factor  is  trailed  by  ShipMax,  followed  by  NumHelo.  In  the  partition  models  for 
MOE  2b,  3b,  and  4b  (Figures  35-40)  however,  ShipMax  becomes  the  most  dominant 
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factor.  Additionally,  for  the  first  time,  the  partition  trees  for  MOE  2b  and  3b  distinguish 
the  effect  of  having  2  UAVs  versus  a  single  UAV  or  no  UAVs,  displaying  an  average 
mean  search  time  drop  from  20  to  9  hours(SD  drop  from  22  to  13  hours),  and  15  to  7 
hours(SD  drop  from  20  to  11  hours). 
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Figure  33.  Partition  summary  (MOE  lb) 


Figure  34.  Partition  tree  (MOE  lb) 
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i  -  Partition  for  0.75  2 
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Figure  35.  Partition  summary  (MOE  2b) 


Figure  36.  Partition  tree  (MOE  2b) 
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Figure  37.  Partition  summary  (MOE  3b) 


Figure  38.  Partition  tree  (MOE  3b) 
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i  -  Partition  for  0.25  2 


0.177  16.199751  484  3  42  4079,53 


Figure  39.  Partition  summary  (MOE  4b) 


Figure  40.  Partition  tree  (MOE  4b) 


The  controllable  factor  metamodels  are  relatively  worse  than  the  full  factor 
metamodels  as  far  as  explaining  the  variability  in  the  NPSS  model’s  output  data.  The 
Rsquare  values  achieved  in  these  partition  metamodels  are  0.334,  0.182,  0.193,  and  0.177 

for  MOE  lb,  MOE  2b,  MOE  3b,  and  MOE  4b  respectively. 
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2.  Number  of  Splits  versus  Rsquare 

To  get  an  idea  of  the  partition  fit,  the  Rsquare  of  each  of  the  “b”  series  MOEs  is 
tabularized  as  a  function  of  CDP  threshold  and  the  number  of  splits  (Table  7). 


Pd  threshold 


0.95 

0.75 

0.5 

0.25 

1 

0.106 

0.052 

0.073 

0.093 

2 

0.296 

0.092 

0.105 

0.116 

3 

0.296 

0.128 

0.141 

0.18 

4 

0.329 

0.182 

0.192 

0.186 

5 

0.346 

0.199 

0.221 

0.209 

6 

0.361 

0.215 

0.246 

0.242 

7 

0.37 

0.231 

0.268 

0.248 

8 

0.37 

0.249 

0.271 

0.255 

9 

0.375 

0.252 

0.292 

0.277 

10 

0.382 

0.253 

0.294 

0.295 

11 

0.389 

0.258 

0.3 

0.296 

12 

0.398 

0.259 

0.31 

0.299 

Table  7.  Rsquare  for  MOE  “b”  series 


Note  that  for  each  MOE  (see  Table  7),  again  as  expected,  the  Rsquare  remains 
relatively  low,  even  after  twelve  splits,  meaning  that  it  is  relatively  difficult  to  predict  the 
performance  of  the  ship  with  the  NPSS  model  using  the  decision  factors  in  the  presence 
of  the  various  noise  factors.  Additionally  when  comparing  Table  7  with  Table  6,  Rsquare 
entries  in  Table  7  are  lower  than  those  in  Table  6,  since  Table  7  uses  fewer  factors  to 
describe  the  same  MOEs. 

C.  COMPARISON:  NPSS  MODEL  WITH  ITALIAN  MODEL 

The  Italian  model’s  measure  of  effectiveness  is  based  on  using  simulation  data 
collected  at  the  end  of  each  iteration  to  estimate  the  probability  of  detection.  Although 
like  the  Italian  model,  the  NPSS  model  runs  from  the  beginning  to  the  end  of  each  design 
point/iteration,  the  average  time  is  calculated  for  specific  CDP  thresholds.  Because  of 
this,  the  MOEs  of  the  two  models  are  different,  and  cannot  be  directly  compared 
analytically;  the  comparison  analysis  will  be  limited  to  comparing  factors  that  heavily 
impact  each  respective  model’s  measure  of  effectiveness. 
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In  order  for  the  Italian  model  and  the  NPSS  model  to  be  compared  effectively,  the 
results  of  the  new  SAR  model  need  to  represent  the  scenarios  in  the  Italian  model. 
Certain  parameters  in  the  NPSS  model  must  be  fixed  or  have  a  limited  range  to  match  the 
Italian  model’s  inputs  (shown  in  Table  6).  Note  the  factor  NumUAV  must  be  set  to  zero, 
since  the  Italian  model  does  not  utilize  UAVs.  The  Sweep  widths  of  the  ship  and 
helicopter  also  need  to  be  set  to  fixed  values,  rather  than  functions  of  multiple  factors. 
The  ranges  of  ShipMax,  NumHelo,  and  Target  need  to  be  reduced  as  shown  in  the  table. 
NumHelo  must  be  reduced  to  a  binary  variable,  since  the  Italian  model  does  not  analyze  a 
multiple  helicopter  scenario.  As  for  the  factor  Target,  it  is  unclear  what  the  size  of  the 
“Fishing  Boat”  is  in  the  Italian  model.  Typically  a  fishing  boat  can  be  characterized  as  a 
“power  boat”  up  to  90  ft.  in  length,  and  so  for  comparison  purposes,  in  the  NPSS  model, 
the  Target  factor  can  be  limited  to  target  numbers  10-15. 


Original  Factors  of  New  Model  Changes  in  Factors  of  New  Model 


ShipSearchSpeed  = 

[0, ShipMax] 

knots 

ShipSearchSpeed  = 

15 

knots 

HeloSearchSpeed  = 

[60,HeloMax] 

knots 

HeloSearchSpeed  = 

70 

knots 

DatumU  = 

[0,20] 

nm 

DatumU  = 

2 

nm 

WindSpeed  = 

[0,80] 

knots 

WindSpeed  = 

20 

knots 

NumUAV  = 

[0,1,2] 

NumUAV  = 

0 

Corrected  W  ship  = 

f(Vis, Target) 

nm 

Corrected  W  ship  = 

2 

nm 

Corrected  Whelo  = 

f(Vis,Target,HeloSearchSpeed,ECF) 

nm 

Corrected  W  helo  = 

5.8 

nm 

ShipMax  = 

[10,60] 

knots 

ShipMax  = 

[22,40] 

knots 

NumHelo  = 

[0,1,2] 

NumHelo  = 

[0,1] 

Target  = 

[1,2,. ..,25] 

Target  = 

[10,...,14] 

Table  8.  NPSS  model  changes 


Fitting  the  Italian  model’s  data  to  a  forward  moving  2nd  order  stepwise  regression 
using  minimum  BIC  as  the  stopping  criteria,  (shown  in  Figure  41)  results  in  an  analytical 
model  with  an  Rsquare  adjusted  of  0.998.  Additionally,  the  regression  fit  suggests  that 
the  biggest  impact  to  the  scenario  is  primarily  due  to  the  presence  of  a  helicopter, 
followed  by  the  distance  (which  corresponds  to  DatumCDR  in  the  NPSS  model),  and 
lastly  due  to  the  ship’s  maximum  speed  (which  corresponds  to  ShipMax). 
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Sum  of 


Source 

DF 
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Mean  Square 
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Model 

8 
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Error 
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Figure  4 1 .  Regression  summary  (Italian  results) 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

55.06786 

0.343677 

160.23 

<0001* 

Helo 

44.823308 

0.10497 

427.01 

<0001* 

Distance 

-0.023643 

0.00105 

-22.52 

<0001* 

MaxSpeed 

0.0884837 

0.009582 

9.23 

<0001* 

(Helo-0.5)’(Distance-125) 

0.0389549 

0.002099 

18.56 

<0001* 

(Distance-125)*(Distance-125) 

-0.000145 

2.424e-5 

-5.97 

<0001* 

(Helo-0.5)’(MaxSpeed-31) 

-0.164386 

0.019165 

-8.58 

<0001* 

(Distance-l25)’(MaxSpeed-31) 

0.0013467 

0.000192 

7.03 

<0001* 

(MaxSpeed-31)*(MaxSpeed-31) 

-0.005513 

0.001964 

-2.81 

0.0054* 

Effect  Tests 

Sorted  Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Helo 

44.823308 

0.10497 

427.01 

<0001* 

Distance 

-0.023643 

0.00105 

-22.52 

:  :  [ 

<.0001* 

(Helo-0.5)’(Distance-l25) 

0.0389549 

0.002099 

18.56 

1 

<0001* 

MaxSpeed 

0.0884837 

0.009582 

9.23 

:  i  :  i 

<0001* 

(Helo-0.5)*(MaxSpeed-31) 

-0.164386 

0.019165 

-8.58 

<0001* 

(Distance-125)*(MaxSpeed-31) 

0.0013467 

0.000192 

7.03 

<0001* 

(Distance-125)*(Distance-125) 

-0.000145 

2.424e-5 

-5.97 

<.0001* 

(MaxSpeed-31)*(MaxSpeed-31) 

-0.005513 

0.001964 

-2.81 

0.0054* 

Figure  42.  Regression  fit  (Italian  results) 
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After  making  the  changes  in  Table  6,  using  an  arbitrary  MOE  (in  this  case  MOE 
1)  and  fitting  the  output  data  to  a  forward  2nd  order  stepwise  regression  using  minimum 
BIC  as  the  stopping  criteria  regression,  the  statistical  metamodel  achieves  an  Rsquare 
adjusted  of  0.922.  Note  that  this  regression  (details  in  Figure  44)  also  suggests  that  the 
presence  of  the  helicopter  has  the  most  impact  on  the  MOE,  followed  by  the  DatumCDR, 
and  then  by  ShipMax,  although  the  directions  of  each  of  the  impacts  are  reversed. 


*■  Response  0.95  2 


zi  Summary  of  Fit 


A\  Summary  of  Fit 

RSquare  0.925756 

RSquare  Adj  0.922221 

Root  Mean  Square  Error  8.26732 

Mean  of  Response  39.49624 

Observations  (or  Sum  Wgts)  111 

A  Analysis  of  Variance 


Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Model 

5 

89486.245 

17897.2 

261.8526 

Error 

105 

7176.600 

68.3 

Prob  >  F 

C.  Total 

110 

96662.845 

<0001* 

Figure  43.  Regression  summary  (NPSS  model) 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

71.304894 

6.143286 

11.61 

<0001* 

ShipMax 

-0.423597 

0.162339 

-2.61 

0.0104* 

DatumCDR 

0.0848067 

0.019413 

4.37 

<  0001* 

NumHelo 

-56.84284 

1.586863 

-35.82 

<0001* 

(ShipMax-31 .5407)*(DatumCDR-1 28.71 1 ) 

-0.009785 

0.004252 

-2.30 

0.0234’ 

(DatumCDR-128.711)*(NumHelo-0. 52252) 

Effect  Tests 

0.1049136 

0.038791 

2.70 

0.0080’ 

Sorted  Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

NumHelo 

-56.84284 

1.586863 

-35.82 

<0001* 

DatumCDR 

0.0848067 

0.019413 

4.37 

<0001* 

(DatumCDR-128.71 1  )*(NumHelo-0.52252) 

0.1049136 

0.038791 

2.70 

0.0080* 

ShipMax 

-0.423597 

0.162339 

-2.61 

0.0104* 

(ShipMax-31. 5407)*(DatumCDR-1 28.711) 

-0.009785 

0.004252 

-2.30 

0.0234* 

Figure  44.  Regression  fit  (NPSS  model) 
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After  comparing  the  first  MOE  to  the  Italian  model,  it  may  be  interesting  to 
compare  the  Italian  model  with  all  four  of  the  MOEs  of  the  NPSS  model.  Table  7  is 
populated  with  regression  data  which  includes  main  effect  coefficients  and  interaction 
term  coefficients  for  all  four  of  the  NPSS  model  MOEs  as  well  as  the  same  data  for  the 
Italian  model,  for  comparison  purposes. 


Italian  Model 

NPSS  Model 

MOE  1(0.95) 

MOE  2(0.75) 

MOE  3(0.5) 

MOE  4(0.25) 

RA2  Adj. 

0.998 

0.922 

0.527 

0.526 

0.520 

#of  Terms 

8 

5 

5 

5 

5 

Main  Effects 

NumHelo 

44.823 

-56.842 

-27.873 

-20.591 

-10.83 

(standard  error) 

0.104 

1.586 

2.96 

2.371 

1.455 

DatumCDR 

-0.0236 

0.084 

0.186 

0.16 

0.106 

(standard  error) 

0.001 

0.019 

0.036 

0.029 

0.017 

ShipMax 

0.088 

-0.423 

-0.73 

-0.699 

-0.425 

(standard  error) 

0.0095 

0.162 

0.302 

0.242 

0.149 

Interaction  Terms 

NumHelo*DatumCDR 

0.038 

0.105 

-0.218 

-0.222 

-0.138 

(standard  error) 

0.002 

0.038 

0.072 

0.058 

0.036 

ShipMax*DatumCDR 

0.0013 

-0.0097 

-0.018 

-0.015 

-0.009 

(standard  error) 

0.00019 

0.0042 

0.0079 

0.0063 

0.004 

DatumCDRA2 

-0.000145 

(standard  error) 

0.000024 

NumHelo*ShipMax 

-0.164 

(standard  error) 

0.019 

ShipMaxA2 

-0.0055 

(standard  error) 

0.0019 

Table  9.  Metamodel  comparison 


It  is  interesting  to  note  that  the  Rsquare  adjusted  value  greatly  drops  going  from 
MOE  1  to  MOE  2,  and  continues  to  remain  around  0.52  for  each  subsequent  MOE.  This 
seems  to  suggest  that  even  though  the  scenario  is  fixed  to  mimic  the  Italian  model,  if  the 
desired  cumulative  probability  detection  threshold  is  relatively  low,  the  variability  in  the 
model  can  be  relatively  high,  meaning  it  would  be  less  useful  for  predictive  purposes. 
Additionally,  from  the  four  MOE  fits  of  the  NPSS  model,  MOE  1  seems  to  replicate  the 
trends  and  fit  of  the  Italian  model  the  best.  The  signs  on  the  coefficients  are  reversed,  but 
this  is  appropriate  because  good  alternatives  for  the  Italian  model  correspond  to  large 
values  of  their  MOE  (CDP),  while  good  alternatives  for  the  NPSS  model  correspond  to 
low  values  of  the  MOEs  (times  to  achieve  a  particular  CDP  threshold). 
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IV.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

From  Chapter  I,  recall  that  the  primary  and  secondary  questions  are: 

Can  a  model  of  SAR  Operations  be  developed  using  Microsoft  Excel  to  show 
potential  mission  effectiveness  of  a  ship  design  concept  with  results  comparable  to  the 
Italian  research  team’s  simulation  results? 

Can  this  new  model  be  improved  through  more  realistic  operational 
representantion  and  explored  using  enhanced  experimental  design  techniques  in  order  to 
provide  broader  insights  than  the  results  from  the  original  Italian  model? 

To  address  the  primary  question,  the  NPSS  model  that  is  developed  throughout 
Chapter  II  is  built  in  a  Microsoft  Excel  file.  Although  the  new  search  model  may  have 
benefitted  from  a  decrease  in  computation  time  if  it  were  to  be  written  in  computer  code 
such  as  Java,  or  C++,  Microsoft  Excel  has  proven  to  be  capable  of  conducting  the 
calculations.  In  Chapter  III,  the  NPSS  model’s  data  is  used  to  build  a  full  factor 
metamodel,  and  then  again  to  build  a  controlled  analytic  model  which  aims  to  minimize 
variability  by  fitting  the  model  in  the  absence  of  noise  factors.  In  both  cases,  the 
metamodels  achieve  relatively  low  Rsquare  values,  which  leads  to  the  conclusion,  that 
the  search  mission  has  a  large  amount  of  variability  associated  with  it.  When  the  NPSS 
model  is  examined  only  in  scenarios  that  mimic  the  scenarios  analyzed  by  the  Italian 
model,  its  output  becomes  much  more  predictable.  As  a  result,  a  metamodel  for  MOE  1 
achieves  an  Rsquare  of  0.922,  and  for  comparative  purposes,  establishes  the  same  main 
effect  ordering  of  importance.  This  shows  that  for  the  restricted  set  of  search  operations 
examined  in  the  Italian  study,  the  output  of  both  the  Italian  model  and  the  NPSS  model 
are  very  predictable. 

As  for  the  secondary  question,  the  experiment  using  the  NPSS  model  incorporates 
many  more  factors  than  the  original  experiment  using  the  Italian  model;  mostly  noise 
factors,  which  do  exist  in  the  operational  environment.  Additionally,  the  option  of  having 
fixed-wing  UAVs  is  incorporated  in  the  NPSS  model.  Also,  the  NPSS  model  allows  for 
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the  analysis  of  multiple  UAVs  and  helicopters  as  opposed  to  the  Italian  model’s  sole 
helicopter  option.  Lastly,  the  NPSS  model  allows  air  search  assets  to  explore  and  search 
independently  from  the  ship,  and  to  search  at  a  range  of  search  speeds,  rather  than  being 
limited  to  a  fixed  speed  directly  dependent  on  the  search  speed  of  the  ship.  These 
additions  allow  the  experiment  involving  the  NPSS  model  to  explore  various  dimensions 
that  cannot  be  analyzed  by  the  current  Italian  model.  When  more  factors  are  varied  in  the 
NPSS  experiment,  it  is  difficult  to  achieve  metamodels  with  a  relatively  high  Rsquare. 
This  means  that  the  way  all  these  factors  combine  in  different  circumstances  is  quite 
complex,  and  not  something  that  can  be  described  with  a  single  simple  equation.  In 
reality,  there  are  too  many  variables  that  can  affect  the  perfonnance  of  a  ship 
configuration. 

B.  RECOMMENDATIONS  AND  FUTURE  WORK 

1.  Italian  Model  Recommendations 

Although  the  output  from  the  experiment  conducted  in  the  Italian  study  can  be  fit 
quite  well  with  a  simple  regression  metamodel  (that  is,  the  metamodel  is  able  to  explain 
much  of  the  variability  in  the  model’s  output  data),  the  results  from  the  NPSS  experiment 
suggest  that  this  will  not  occur  when  realistic  noise  factors  are  allowed  to  vary. 
Performing  a  similar,  large-scale  experiment  using  the  Italian  model  would  provide  the 
users  with  a  more  realistic  representation  of  the  uncertainty  associated  with  search 
missions. 

Additionally  the  Italian  experiment  may  want  to  incorporate  time  either  as  a 
separate  MOE,  or  to  join  it  with  the  current  MOE,  in  order  to  be  more  operationally 
relevant.  The  analysis  shows  that  the  importance  of  certain  factors  increase,  while  others 
decrease  as  a  function  of  cumulative  probability  detection  threshold.  Since  by  definition, 
the  cumulative  probability  of  detection  is  a  monotonically  increasing  function  of  time, 
using  time  as  a  threshold  rather  than  the  cumulative  probability  of  detection  will  yield 
similar  results. 
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2. 


NPSS  Model  Future  Work 


a.  Non-Rect angular  Search  Box 

Instead  of  using  the  rectangular  search  box  incorporated  by  the  Italian 
model,  the  shape  of  the  box  may  be  modified  to  search  more  efficiently  (shown  in  yellow 
in  Figure  45);  only  in  those  areas  most  likely  for  the  target  to  be  located.  Since  the 
coverage  factor  used  in  the  ICL  uses  the  value  of  the  search  area  A  directly,  the 
modification  is  a  seamless  substitution  of  the  new  value  of  the  area. 


Figure  45.  More  efficient  search  box  (After  Anghinolfi  et  ah,  2011) 
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b. 


“Rescue”  Aspect  of  Mission 


The  NPSS  model  currently  does  not  account  for  the  rescue  process  of  the 
mission.  It  primarily  focuses  on  the  search  and  location  of  the  target.  Unless  a  UAV  is 
capable  of  rescuing  the  survivor(s),  once  a  UAV  locates  a  target,  a  rescue  time  delay 
must  be  incorporated  to  account  for  either  the  closest  helicopter  or  the  ship  to  move  from 
its  current  position  to  the  target. 

c.  SAR  Hazards 

Depending  on  whether  the  target  is  submersed  in  water,  and  an  additional 
noise  variable  to  represent  water  temperature,  the  plot  on  Figure  46  can  be  used  to 
detennine  a  successful  rescue. 


WATER  CHILL  WITHOUT  ANTIEXPOSURE  SUIT 


Figure  46.  Water  chill  without  anti-exposure  suit  (From  Office  of  the 
Chief  of  Naval  Operations,  1997) 


d.  Helicopter/UA  V  Refueling/Maintenance 

In  addition  to  rescue  time  delays,  refueling  delays  can  be  incorporated  for 
helicopters  and  UAVs  conducting  the  search.  Additionally,  helicopters  and  UAVs 
undergo  wear  and  tear,  and  have  scheduled  and  unscheduled  maintenance  periods  that 
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can  be  incorporated  in  the  NPSS  model.  A  new  count  variable  may  need  to  be 
incorporated  to  keep  track  of  how  many  search  entities  are  undergoing  maintenance. 

e.  Additional  Noise  Variables 

There  are  a  few  other  noise  factors  that  can  be  incorporated  in  the  NPSS 
model  for  future  work.  Some  are  readily  available  in  the  National  SAR  manual, 
specifically  flight  altitudes  of  aerial  search  entities,  and  sweep  width  correction  factors 
associated  with  the  target.  Both  of  these  noise  factors  can  affect  the  overall  sweep  width 
of  the  search  entity,  and  in  turn  the  perfonnance  of  the  ship  configuration. 

C.  SUMMARY 

This  thesis  provides  a  link  between  physical  ship  design  factors  and  operational 
effectiveness  of  the  SAR  mission.  The  results  contribute  to  a  larger  project  that  aims  at 
developing  a  methodology  for  evaluating  the  operational  effectiveness  of  OP  Vs  for  a 
variety  of  missions  before  proceeding  to  the  detailed  design  of  these  units. 
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